AI companies are starting to find out that bigger isn’t always better.
Driving the news: The likes of OpenAI and Google are reportedly seeing diminishing returns on work creating their next, more powerful AI models. Developers assumed throwing computing power, chips, and talent at their models would keep the rate of improvement going steady, but their next models aren’t yet seeing as big of a step up in performance.
- For what it’s worth, a cryptic tweet by OpenAI CEO Sam Altman seemed to be an attempt to refute the idea that they’ve hit “a wall.” Reports also claim the company is readying an AI agent for release in the new year.
Why it’s happening: It could simply be because companies are trying to move faster than their resources or technological capabilities will allow. But AI is also only as good as the data it is trained on — and with most of the data available having already been scraped, there are few new inputs to make models better.
Why it matters: The stuff generative AI currently creates — whether it’s text, audio, visuals, or coding — is far from perfect, but companies have said they are laying the groundwork for something truly impressive down the line. The longer that takes, the greater risk of losing impatient customers and investors alike.
- It also casts doubt on when they might achieve their goals for artificial general intelligence, or if it is even possible.
What’s next: One way companies are trying to overcome this is by putting less of a focus on the training stage of models and more on inference — how the model behaves after it has been released and “reasons” after being exposed to new data. This could be a boon for firms like Untether, which makes chips specialized for inference.