I find when driving in Boston that pedestrians who are just about to jaywalk often make brief eye contact before darting out, so it’s complicated.
I assume that brief eye contact is one if the last acts of their life before they’re run down.
Boston driving seems to be the real world example of the old saw:
Don’t like my driving? Get off the sidewalk.
Still not quite ready.
I was prepared for this to be exaggerated, and certainly that’s how she was for the first couple of minutes, but I agree that was pretty funny. 8+ years of progress!
This is surprising. I think self-driving trucks provide both a simpler technical issue (you can limit the driving to Interstates and similar roads plus a limited number of origins/destinations) plus a stronger economic justification (getting rid of the cost of the high priced truck drivers and running trucks continually except for maintenance, loading/unloading and refueling…).
I saw the headline and came to the same conclusion as you …
they pulled the wrong plug IMHO …
being the interstate simplicity and the 24/7 use of a rather expensive capital good like a truck and docking stations the main driver (hey! :o)
Also, the robo-truck would integrate to the whole “fully robotized distribution centers” nicely …and could even put new efficiencies in JIT
- robo-trucks arrive
- are robo-unloaded - robo-stored
- pallets are robo-rescrambled according to orders and robo-loaded
- and then been delivered by manned trucks to supermarkets/bodegas/7-11s/etc…
makes way more sense to me - and also seems like a way larger cake to take than a taxi-operation
We’d probably have to dig into the financials to get a full picture, here.
Daimler Trucking NA has their own in-house autonomous vehicle group that they’re still touting on their website (no mention of Waymo on that page).
UPS had partnered with Waymo as a secondary option, but appear to still have their relationship with TuSimple.
Ryder is also partnering with TuSimple but seems to be focused on becoming a services provider for autonomous vehicle operations.
JB Hunt doesn’t seem to have a backup but maybe their financials just don’t support the slow progress
This is from a cursory search and I’ve not dug deep into any of these companies, but it may be that autonomous long-haul truck funding has dried up for Waymo specifically, but is still going strong across the industry. The other companies involved have been focused on autonomous trucking, whereas Waymo has been focused on ride hailing, so it may be that Waymo wasn’t actually bringing much value to these companies.
eta: That’s all to say, the decision to focus on raid hailing may have been forced on Waymo by their partners, and not the other way 'round.
Very few (no?) distribution centers connect directly to an interstate via dedicated ramps.
So a robo-truck still needs to be able to navigate ordinary surface streets well enough to not kill people or get stuck in places where it’s big and the roads or surrounding cars are not. And it needs to be able to operate in truck stops which are hotbeds of chaos with no easy markings to see and follow. And with rather tight tolerances for parking, fueling, etc.
In all, I don’t really see how long-haul robo-trucking is easier than robo-car-ing. Yes, the mileage mix is more interstate-heavy. But that doesn’t alter the minimum capabilities required. It just alters how often each capability will be used.
I don’t know if this is still true, but for a long while the conversation was around creating new hubs near the interstate where they could both offload to local conveyance and gas up. That never seemed particularly efficient to me – I think people have a poor sense of distance – but it was a control for some of the bigger complications.
I suspect one of the reasons to pull back is that the California legislature has become increasingly antagonistic toward Robo trucks, and I have been insisting on having operators involved.
California’s AV bill advances as industry, legislators clash on safety issues | Transport Dive.
Only 3 years to go and the first decade since the start of this thread will be passed.
how much of the LLM-KI can cross-fertilize self-driving? … or are those completely different logical-skill-sets? are there meta-structures or knowledge that could carry over from one field to the other?
I really remember my surprise how good stuff like Chat-GPT has become in a really short time-frame…
It went - in terms of practical usability - from useless to pretty much as good as an above avg. educated person could do it in what felt like 6 months …
any thoughts?
One quick thought.
A Chat AI can be trained on the entire corpus of what’s on the internet, plus any / all ebooks. Plus for outfits like Google and Microsoft, every email that passes through gmail.com or Hotmail / Live / Outlook.com.
The corresponding training dataset for driving would be a video / LIDAR feed from every car on the road. Part of the problem with the AI cars to date is that despite a reltive handful of years driving a relative handful of cars on a relative handful of roads, they have come nowhere near getting a large enough corpus of unusual cases, much less edge cases. So the AI car does dumb things in slightly non-routine situations, and really dumb things in utterly non-routine situations. And often can’t even tell slightly non-routine from normal.
If we didn’t mind the ensuing early carnage, we could simply loose a few million fully telemetered cars on the road with current best-of-class AI’s in control and allow their one central “mind” (i.e. driving model) to learn by doing over the next few years of calendar time and few million vehicle-years of driving.
As I often joke about people parallel parking, there’ll be a lot of Braille method driving going on. But there will be a lot of learning too. 
One place where it may help is reading and understanding road signs. If I give chatgpt the input “maximum speed 55 MPH vehicles over 20,000 lbs GVW” it may understand what is meant. Similar “right lane closed ahead” or an overhead warning sign that says “debris in left lane”.
As output an essay about truck speed limits in various conditions would be pretty useless, so the output would have to be adjusted to something that could be understood by the driving model: “we are not a truck, do not set speed to 55 MPH;” “move out of the left lane”; etc.
My guess is the primary impact language models will have on self driving is just the general increase in the technology surrounding AI models. Successful AI technology in one field that can be commercialized will drive further development, which will benefit other fields.
Or do like Tesla is actually doing in reality: every car they sell (either with the FSD package or not) is permanently in a kind of ghost mode, and automatically set to record unusual events and upload them to Tesla. Tesla also has the capability of changing the triggers on demand, so if they realize there is something that they are undertrained for, they can intentionally seek out additional examples across the fleet.
Andrej Karpathy has talked about this feature in a number of his FSD talks. IIRC, one early example was vehicles with a rear bike rack. The car was detecting the bicycles as separate vehicles on the road. But they could easily add a trigger and immediately got zillions of additional examples to train on. Their fleet is so large now (millions) that they should be able to do the same with even highly unusual events.
Another thing in common use is to use simulated data. Instead of actual real-world video, you train on rendered data. Once you identify a particular edge case, you can then create a bunch of variations on the theme.
It’s non-trivial, because if you are not very careful with the simulation, the AI model might pick up on subtleties that are in the training data but not reality. So they have to do things like simulate camera noise, and camera distortion, and so on. Not to mention having a very realistic scene. It’s not a perfect replacement for reality, but it can partially fill in the gaps. You can also use adversarial AI, so that you pit two systems against each other, where one learns to create ever more accurate simulations, while the other detects the difference between simulations and reality. If an AI model specifically trained to pick up the difference cannot, then the simulated data should be safe for other uses.
Good to know. Thank you. I suspected Teslas were “phoning home” a lot, but wondered just how much detail and how often.
People who pay attention to their WiFi networks often find that the car has uploaded several gigabytes of data in the course of a few days. It’s not entirely predictable–which I suppose is consistent with it being mostly triggered by unusual circumstances.
Here’s the bit where Karpathy was talking about it:
Basically consistent with what I said. Though that was 4 years ago and I’m sure they’re dealing with much finer edge cases than bike racks today.
Cruse car halts for 20 minutes in construction zone until Cruise sends someone out. Apparently this is not at all uncommon–so San Francisco is resisting massive expansion of self-driving taxis which Cruise and Waymo are planning:
Fortunately, Musk is now apparently claiming that they’ve figured out AGI (or ‘some aspects of’ it, whatever that may mean), and that the car has a mind as a result (or maybe a little bit of one), which if true would be a horribly dystopian development. (I’m just going to assume I’m missing some context on these remarks.)
That said, I think full self-driving has been conjectured to be AI-complete anyway, so one probably should not think of it as distinct from achieving AGI.
FSD v12 should be interesting. It’s the first end-to-end AI system. Previous versions were segmented and had hard-coded parts.
Obviously, Musk’s statements should be taken with a large grain of salt. And in any case, there’s enough disagreement about what constitutes AGI that you can’t conclude much about a statement like that. I’m fairly sure you would disagree that LLMs demonstrate some aspects of AGI, given your claims that they don’t understand anything at all, but I think they do (which isn’t to say that’s the route to a full AGI, either).
I’d guess that their system is showing interesting emergent behavior. High-level generalization. Doing the right thing even in cases far outside the training set. That may not be something they’ve seen before, and are inferring that some near-AGI thing is going on.
I mean, I can set up a road network that requires solving a large 3SAT system to progress through, so is driving NP-complete also?
If Level 5 is defined to work in any environment where a human is capable of driving, then most humans are not Level 5–they’re Level 4.9 or something. There are roads I won’t go on because I don’t have the skills to progress safely. I expect the same to be true of self-driving, but the set of roads where it can’t drive will be different.