Nvidia is known for making chips, but a new AI model that runs on them could be what changes the course of the industry.
What happened: Nvidia released NVLM, its own family of generative AI models capable of analyzing images, producing text, coding, and solving math problems on par with the likes of GPT-4o and Meta’s Llama. Nvidia will also make the models open, making their training data and code publicly available.
- The flagship NVLM-D-72B model has 72 billion parameters — the more parameters a model has, the more complex and detailed tasks it can complete.
Why it matters: Opening the models breaks from convention in the industry, where the inner workings of top-of-the-line models have been closed off. That access could accelerate progress for researchers and developers. But it could also force major developers to reconsider their approach.
- Nvidia may be in a better economic position to open its AI than OpenAI and Google, which are trying to get revenue from their massive investments in AI — much of which went to buying chips from Nvidia.
Yes, but: OpenAI models come with safety and ethical risks, since coders could remove guardrails that keep them from generating things like deepfakes, hacking code, or instructions for building dangerous devices.
How it works: The other thing about NVLM that could be very influential is the way it was designed. Instead of processing data and requests one way, it takes a hybrid approach that combines multimodal techniques.
- Many models perform worse at text-based tasks after multimodal training, as more types of inputs create more info to sort through. NVLM, on the other hand, seems to get better by taking the best processing approach for the task at hand.
Zoom out: Other big tech companies are pushing ahead on developing bigger and better AI models of their own. Google has reportedly made progress on performing reasoning tasks like math and coding as it tries to catch up with OpenAI’s o1 release last month.