# Open AI's latest wonder: text to video with Sora **URL:** \ **Category:** Cafe Society\ **Tags:** ai\ **Created:** [February 16, 2024, 9:36am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648 "2024-02-16T09:36:30Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![Atamasama](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/atamasama/32/12961_2.png) [@Atamasama](https://boards.straightdope.com/u/Atamasama)\ **Post date:** [February 18, 2024, 2:46am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/21 "2024-02-18T02:46:49Z")
Gah, forgot I was in Cafe Society. ☹ ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [February 18, 2024, 1:07pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/22 "2024-02-18T13:07:04Z")
Or that the phone isn’t the camera that would have been recording the image. There are half a dozen ways to keep cameras out of reflections, so the footage isn’t necessarily unnatural in that regard. ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [February 18, 2024, 9:52pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/23 "2024-02-18T21:52:03Z")
Gemini 1.5 can do video, and it now has a context window of a million tokens. It can ingest an entire movie and make sense of it. In fact, one of the demos was that they gave it a copy of a n entire silent film with no text, no subtitles, and no description of the film. The AI was able to accurately summarize each scene and the whole film, describe the themes and plot, etc. Astounding how far video has come in just a few months. ---
**Author:** ![Lantern2](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lantern2/32/3010_2.png) [@Lantern2](https://boards.straightdope.com/u/Lantern2)\ **Post date:** [February 18, 2024, 10:59pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/24 "2024-02-18T22:59:51Z")
This is the [Gemini](https://www.youtube.com/watch?v=wa0MT8OwHuk) demo and yeah it’s quite astonishing. It will probably revolutionize police/security/military video analysis very soon among other things. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [February 18, 2024, 11:09pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/25 "2024-02-18T23:09:37Z")
> [@Sam\_Stone](#): > > Astounding how far video has come in just a few months. This was state-of-the-art text to video in March of 2023. At the time all the rage was making _x_ eating _y_ (typified by Will Smith eating spaghetti). These are some of the very few tries I made. (Note that the AI thought all video should have a Shutterstock watermark.) A cat eating a frog [![](https://img.youtube.com/vi/MJjfrpvgR6c/hqdefault.jpg "ancient ai video 2") ](https://www.youtube.com/watch?v=MJjfrpvgR6c) Taylor Swift eating a cat [![](https://img.youtube.com/vi/MS3WChZJEb0/hqdefault.jpg "ancient ai video 1") ](https://www.youtube.com/watch?v=MS3WChZJEb0) Bojack Horseman eating honeydew [![](https://img.youtube.com/vi/eoy7JPvI2po/hqdefault.jpg "ancient ai video 3") ](https://www.youtube.com/watch?v=eoy7JPvI2po) ---
**Author:** ![Sam\_Stone](https://avatars.discourse-cdn.com/v4/letter/s/ecccb3/32.png) [@Sam\_Stone](https://boards.straightdope.com/u/Sam_Stone)\ **Post date:** [February 18, 2024, 11:15pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/26 "2024-02-18T23:15:10Z")
Well, that could be Bojack eating a Honeydew, filmed woth a macro lens… Yeah, the latest videos re actually production quality for things like commercials and corporte videos. Second unit directors shooting B-roll and commercial directors are probably in trouble. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [February 18, 2024, 11:31pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/27 "2024-02-18T23:31:35Z")
If you want to try text-to-video that is better than my above examples but not as good as Sora, you get a number of free seconds at Runway. > **[Runway - Advancing creativity with artificial intelligence.](https://runwayml.com/)** > > Runway is an applied AI research company shaping the next era of art, entertainment and human creativity. I used most of my free time on image-to-video, which takes a still image and tries to convert that to video with no descriptive prompt at all. Results for that can be a little okay, but fall apart fast even in the short time available. I’d love to see what Sora could do with image-to-video. Here’s some of my Runway results coverting some of my text-to-image images into video. [![](https://img.youtube.com/vi/Oe-VM-bgI1U/maxresdefault.jpg "RunwayML Gen2 test clip compilation") ](https://www.youtube.com/watch?v=Oe-VM-bgI1U) And here are some using Stable Video Diffusion, which so far does only image-to-video, no text-to-video. [![](https://img.youtube.com/vi/FAZUe4LfmUU/maxresdefault.jpg "Stable Video Diffusion sample clips") ](https://www.youtube.com/watch?v=FAZUe4LfmUU) ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [February 21, 2024, 12:19am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/28 "2024-02-21T00:19:32Z")
Some more AI video > **[Everyone in this video is AI generated](https://boingboing.net/2024/02/20/everyone-in-this-video-is-ai-generated.html/amp)** > > A new era of fake videos is here, raising questions about the future of privacy and security. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [February 21, 2024, 12:21am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/29 "2024-02-21T00:21:51Z")
Also, this happened recently. > **[Scammers Just Tricked An Employee With AI-Generated Copies Of Their Coworkers](https://uk.news.yahoo.com/dystopian-news-scammers-just-used-215804318.html?guccounter=1)** > > An AI-generated representation of the company's chief financial officer was convincing enough to do real damage, Hong Kong police said. ---
**Author:** ![Maserschmidt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/maserschmidt/32/18829_2.png) [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)\ **Post date:** [February 21, 2024, 1:14am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/30 "2024-02-21T01:14:31Z")
> [@Darren\_Garrison](#): > > Some more AI video They’re still struggling with hands, but impressive nonetheless. [![](https://i.imgur.com/zQkYYky.jpg) ](https://i.imgur.com/zQkYYky.jpg) [![](https://i.imgur.com/3S5vntB.jpg) ](https://i.imgur.com/3S5vntB.jpg) ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [February 21, 2024, 7:44am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/31 "2024-02-21T07:44:15Z")
> [@Darren\_Garrison](#): > > At the time all the rage was making _x_ eating _y_ (typified by Will Smith eating spaghetti). Awesome > **[Will Smith parodies viral AI-generated video by actually eating spaghetti](https://arstechnica.com/information-technology/2024/02/will-smith-parodies-viral-ai-generated-video-by-actually-eating-spaghetti/)** > > Actor pokes fun at 2023 AI video by eating spaghetti messily and claiming it's AI-generated. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [March 1, 2024, 11:11am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/32 "2024-03-01T11:11:29Z")
Another day, another mind-blowing AI video demo. > **[Alibaba's AI video generator just dunked on Sora by making the Sora lady sing](https://mashable.com/article/alibaba-emo-ai-facial-animation)** > > Alibaba calls it 'EMO,' and it sure is. The video embedded in the article: [![](https://img.youtube.com/vi/wtcSZdHZne4/maxresdefault.jpg "Alibaba presents EMO AI - All Demo Clips Upscaled to 4K") ](https://www.youtube.com/watch?v=wtcSZdHZne4) ---
**Author:** ![Johnny\_Bravo](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/johnny_bravo/32/493_2.png) [@Johnny\_Bravo](https://boards.straightdope.com/u/Johnny_Bravo)\ **Post date:** [March 1, 2024, 12:28pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/33 "2024-03-01T12:28:56Z")
Ew, no. Those are deeply on the uncanny valley. Impressive technically but not nice to watch. ---
**Author:** ![Pardel-Lux](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/pardel-lux/32/20182_2.png) [@Pardel-Lux](https://boards.straightdope.com/u/Pardel-Lux)\ **Post date:** [March 1, 2024, 1:57pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/34 "2024-03-01T13:57:15Z")
Same applies to the reference images IMO. Most of them anyway. Why did the company choose those? Are they supposed to express normality? Or do they choose slightly weird to hide the evident weirdness of the generated videos? ---
**Author:** ![Maserschmidt](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/maserschmidt/32/18829_2.png) [@Maserschmidt](https://boards.straightdope.com/u/Maserschmidt)\ **Post date:** [March 1, 2024, 3:37pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/35 "2024-03-01T15:37:24Z")
Most of those would have fooled me at a casual glance. Hepburn and young DiCaprio were definitely both uncanny valley for me, however. Impressive technology either way. ---
**Author:** ![nate](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/nate/32/13085_2.png) [@nate](https://boards.straightdope.com/u/nate)\ **Post date:** [March 2, 2024, 11:43am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/36 "2024-03-02T11:43:51Z")
I don’t think I’ve been more impressed with technology in my life. In fact, watching the YouTubes of the videos it creates is causing me to see the world in the same way as if I’m watching an AI created scene. How the fuck does it get the physics 99.9% right?? That’s mind-blowing. I predict that in the not so far future, a matter of months, that the AI will be able to play our human emotions like a fiddle. And we’ll know it but can’t really do much about it… and then kind of lose of confidence that we know anything is “real”, that we are all inferior to what our computers can create now. ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [March 2, 2024, 2:57pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/37 "2024-03-02T14:57:52Z")
And I’ll just remind everyone here to not put too much stock in released demos. If they made 500 attempts and got one that looked good and 499 eldritch horrors, they’re only going to release the one that looked good, and won’t mention the signal to eldritch ratio. ---
**Author:** ![wolfpup](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/wolfpup/32/10618_2.png) [@wolfpup](https://boards.straightdope.com/u/wolfpup)\ **Post date:** [March 2, 2024, 9:57pm UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/38 "2024-03-02T21:57:27Z")
> [@Chronos](#): > > And I’ll just remind everyone here to not put too much stock in released demos. There’s probably a fair amount of truth to that, and the released demos are undoubtedly the best so far. But it’s also worth noting that the diversity of the different scenes and objects suggests that this isn’t just a one-trick pony, but apparently an extremely versatile tool with a very broad visual knowledge of the world and its physical dynamics. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [March 3, 2024, 1:41am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/39 "2024-03-03T01:41:39Z")
I also suspect that with videos of weird lengths (27 seconds or 52 seconds, for random made-up example) instead of the full 1 minute available, they were originally made full length but something went weird before it finished. ---
**Author:** ![Darren\_Garrison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/darren_garrison/32/92_2.png) [@Darren\_Garrison](https://boards.straightdope.com/u/Darren_Garrison)\ **Post date:** [March 15, 2024, 12:51am UTC](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648/40 "2024-03-15T00:51:34Z")
New video generator. Not as good as Sora, and only makes 2 second clips, but it is free to try. Here is my first try, a clip of [Bread Climp](https://boards.straightdope.com/t/digital-art-creator-algorithm-website/962081/2120). [Previous page](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648.md?page=1) [Next page](https://boards.straightdope.com/t/open-ais-latest-wonder-text-to-video-with-sora/997648.md?page=3)