OpenAI’s video generator brings two words to mind: “wow” and “uh-oh.”
What happened: On Thursday, OpenAI released a collection of videos created by its new tool, dubbed Sora. The AI model creates videos up to a minute long with text prompts, but can also animate still images and extend existing video clips.
- This batch — which OpenAI claims were not modified — includes cartoonish videos, like a dancing kangaroo and a giant cloud monster.
- But there are also some very realistic and detailed ones, like a woman walking through Tokyo, puppies playing in the snow, and a crowded Lunar New Year parade.
- Sora still has weaknesses, such as simulating more complex physics, mixing up directions, or maintaining cause and effect (like a cookie remaining whole after a bite was taken out of it).
How it works: Unlike some similar tools, Sora generates the entire video at once, instead of frame by frame. This allows for videos with multiple shots and more consistent imagery — subjects can leave and re-enter the frame without becoming a foot shorter or gaining an extra finger.
Why it matters: These videos are really good, even if not entirely free from defects. The examples (that made it into the public, anyway) are free of many tell-tale signs of AI-generated videos, like oddly moving eyes, strange lighting that makes subjects look like they’re glowing, or “flickering” hair and fur.
Yes, but: There’s a sense of dread that comes with watching AI-generated videos this realistic as trouble with misinformation and deepfakes continues to mount.
- Aside from a select group of artists, Sora is currently only available to a “red team” that will assess it for potential risks, which OpenAI says will be addressed before a broader release.
Zoom out: Google released a first look at Lumiere, its own text-to-video model, last month. Like Sora, it generates the entire video at once, but Lumiere currently only makes low-resolution, five-second clips. It also seems to be better at making cartoony and artistic videos than realistic ones.