Tech companies are continuing to pump out bigger AI models — but will they actually work better?
What happened: Google launched Gemini 2.0, a new version of its AI model that is setting the stage for what many companies in the space believe is the next stage for growth: AI agents that can act on your behalf.
- Agents that Google has shown so far include ones that can find and fix bugs in code, play video games, and use your web browser or other Google apps.
Catch-up: Last week, OpenAI made its o1 model widely available and launched a US$200/month Pro plan that includes extra computing power to help the “reasoning” model with more complex tasks like coding, science, and creating step-by-step instructions.
- o1 is believed to be laying the groundwork for OpenAI’s own agents, as well as more advanced, human-like artificial general intelligence.
Why it matters: There’s growing skepticism about whether the “bigger is better” approach to AI is going to deliver results — namely, that AI or AI agents will ever be useful enough for enough people to want to use them.
- Even though companies are continuing to throw money, talent, and computing power at them, LLMs aren’t getting smarter as fast as they used to.
- For what it’s worth, Google DeepMind CEO Demis Hassabis actually acknowledged the diminishing returns, while OpenAI’s Sam Altman has expressed skepticism.
Zoom out: This is not just about Big Tech taking a reputational hit when it falls short of PR spin or AI delivering humorously incorrect results — it’s about the whole AI bubble bursting when customers can’t find effective use cases to justify the cost.
Big picture: Last week, business-focused AI company Cohere told investors that it was breaking from its peers and not investing in building bigger LLMs. Instead, it was focusing on smaller models tailored to serving a customer’s needs really well.
- With Cohere’s clients constantly generating new owned data from their own operations, its models could also be constantly refined and updated.
Bottom line: Big or small, some believe there are limits to transformer models, the main approach to AI today. A transformer turns text, images, and audio into data, and uses the context of where it has seen similar data in training to essentially guess what a user wants. And when you’re dealing with probabilities, you can never know anything for certain.