Why putting AI on your phone is a big deal

The next phase of the AI boom may not be about what it can do, but where it runs.

Driving the news: Apple is reportedly negotiating to have Google’s Gemini model power AI features on iPhones. A deal isn’t done, but Google is a logical partner, since Gemini has a slimmed-down Nano version optimized to run directly on devices — something Apple seems to be keen on.

• Apple recently acquired DarwinAI. One of the Canadian startup’s services is making AI run more efficiently, which would make it perform better on smaller devices.
   
• This week, Qualcomm launched a new chip that will let mid-priced mobile phones run on-device AI.

Catch-up: Even if you use AI with your phone, it’s not always on your phone. Other than simpler tasks, most processing happens at data centres that can handle AI’s heavy computing needs, accessed through the cloud. But specialized models and chips could change that.

• Keeping everything in your device means AI can be faster and work offline.
   
• Not sending data to a remote server is more private, but also opens up the possibility for an AI to personalize results based on sensitive info secured in a phone or laptop.
   
• The trade-off is that on-device models may not be as “smart,” and are best suited to things like basic photo editing, transcribing meetings, and writing message replies.

Why it matters: Sticking AI into phones could help close the sizable gap between the few companies that have become industry leaders and the competition.

• Nvidia has the market cornered on AI training and data centre chips, but Qualcomm has a leg up in mobile devices — besides ones from Apple, which makes its own chips.
   
• Other than being in iPhone users’ pockets, Google has an edge on OpenAI because GPT-4 does not have a cloud-free, mobile-optimized version (though the rumoured GPT-5 could change that).

Yes, but: On-device AI is most applicable to everyday consumers, who aren’t exactly clamouring for it — 22% of Canadians use it outside of the workplace. Enterprise tasks are the big driver of AI use, and those — at least for now — need cloud computing to be useful.