Fortunately it’s completely optional. you can turn the agent on or off right as a toggle switch in the prompt box and still have the traditional workflow. you can also click the expand button and get a side panel that has a continuous dialogue with the agent over the course of the project. It doesn’t necessarily just generate prompts either. You can ask it to moodboard or evaluate what you’ve already made or suggest new directions and edits or whatever. So it’s a new tool that doesn’t take away from the old workflow if that’s what you want.
The “tools” section itself looks pretty wild. I haven’t played with it yet.
Midjourney’s interpretation of the same prompt always came up with super elaborate flying cruise ships.
On the one hand, I kind of like the fantastical vibe there… but I think it’s ruined by the plane-parts still being streamlined, even though the party decks obviously ruin the aerodynamics. If you’re going to be unaerodynamic, then embrace it fully.
So I used the agent built into google flow to tell it to create me rube goldberg devices. and it correctly understood the assignment and described in words the design of a perfect short goldberg run. and then it feeds that prompt to the actual video generator… which has no idea what to do with it. It creates an authentic looking rube goldberg device. it looks the part. and then the engineering and physics are just nonsense. Which is still a funny enough result.
It’s basically like prompt hacking to break a video generator. it knows what a rube goldberg device looks like… but it likes things that make sense. things that it’s familiar with, things that have an obvious cause and effect. rube goldberg devices deliberately defy our sense of order and utility and design so they baffle the AI
Other than the oversized entrance into the garage, it’s not much wackier than many smaller suburban two-stories cramming too many “features” into insufficient space.
I do like the roll-down garage door which lowers to wall off the toolboxes and workbenches from the rest of the garage. That’s one way to keep those damned kids out of Dad’s tools.
I’m thinking about moving to Italy and I have a few areas picked out as good candidates. I decided to see what google would give me with the following prompt:
“A drone shot high over Trani Italy, looking down, moving across the town center towards the harbor” using the 3.5 Flash Omni video generator.
What I got was actually far more accurate than I expected. Video generators are like image generators – they synthesize trillions of pieces of data in their training data into weights. They’re not Google Earth, they don’t have a reference map of the whole world. Some famous places get accurately represented the same way the Mona Lisa gets accurately represented in image generators - it’s such a dominant force in its training data that it knows that precise image or close enough. But you would expect a relatively obscure town like Trani Italy to have some data (because it’s beautiful) but not anywhere near the famous areas in Italy that are far more well documented.
But it’s surprisingly accurate. It gets some details wrong. The church on the right of the video is actually a famous church that would appear outside of this field of view to the left. Are the exact placements of the buildings and roads precisely accurate? Not exactly. But it absolutely nails the vibe. Those look like real Apulian buildings, construction materials, design, the way the streets are laid out. It correctly gets the series of narrow alleys leading to the harbor. It gets the shape of the harbor right. The relatively size of the harbor, buildings, and streets and their relative positions. It really feels like Trani and not “generic southern Italian town”
For comparison, here’s the google maps 3d view of that same area – wider and higher up for context. The red circle is roughly where the drone view is. If you compare them to the video, it’s pretty close. You can definitely see a few places where it goes wrong, but what’s impressive is that it’s about 80% right. Which is plenty close enough to get a real feel for the vibe of a place.
I assumed that that was the family’s surname, with 1998 probably being their wedding date. And it’s a welcome mat, not a plaque.
EDIT: Looking closer, there are some definite issues with the stairs. First, where’s the bottom of them? Second, the top bannister is narrower than the door, so there must be a weird jag in the wall there. Third, parts of the door look open, and parts look closed. Fourth, why is there even a door there at all?
I am impressed that the three things with text on them all seem to be right (and also look like the sort of thing you’d see in a suburban home).
Bingo. Their wedding date was undefined in the story, but the teenager’s birthdate of 2009 was explicitly defined. But a welcome mat, in front of a garage access door? And not outside? And why is she bringing the bike into the house when the story said she’d naturally put it away in the garage before entering. Grrr.
Yeah. It was mostly the arrangement at the top of the stairs that threw a lot of flags for me. The younger sister had popped out of her room when the older sister returned from her bike ride.
This latest ChatGPT image generator is a LOT better at text in images now. Sometimes throws too much in, but this one had a reasonable amount. And it picked up on how this family are churchgoers, mentioned only once very early on in the context.
Photo of a scene at a supermarket checkout. The white female cashier is lifting the rapper 50 Cent over the counter and handing him to a startled (beardless) white male customer. Iphone 15 photo. 16:9
Ah, I was wondering why you bothered to specify “beardless”.
The last one might be less “bearded” and more just “lazy about shaving”. I look pretty close to that over the summer when I stop caring as much. One of the good things to come out of covid is that fashions are now a lot more tolerant of facial hair, at any level.
So I dug into it a little bit and the documentation suggests that google is actually using its own mapping knowledge or at least an image search when making these videos. That’s part of being a grounded model – it can reference internet data in its creation (live internet data, not just training set weights). Which is a uniquely google advantage because it has such detailed maps of the world they can inform image or video generation. I tried to create a similar video with kling and it wasn’t very good .It basically generated a generic Tuscan coastline (famous, but very different from Puglia in general and not close to any particular city). So I think it’s plausible that google’s system genuinely has an advantage here.
Edit:
Seedream 2.0’s video did better than Kling but worse than Google. It’s recognizably Pugliese and has a famous Trani landmark, the church, though it placed it in the wrong place and separated it from its tower. The harbor shape is all wrong. I would say it constructed some real elements of Trani with no actual geographic knowledge besides that it’s a coastal city.
