In the meantime, human beings will effectively be crash test dummies helping to accumulate all that information.
You have seen teenage boys drive before, right? We put a couple million of those knuckleheads on the road every year, all of them with zero experience and immature frontal cortexes.
Yeah, but when I saw a teenage boy run over a friend of mine and pin her under the car, he understood not to drive off with her under there. So, score one for teenage boys.
I always describe the Tesla FSD as being a teenage (or student) driver. Regardless of perception and reflexes, decision making and situational awareness are terrible.
- The car in front is going 3MPH under our desired speed? Let’s move to the left and pass it less than a mile before we need to make a right turn!
- That car in the other lane behind us is not actually blocking us from changing lanes. It will be in about 15 seconds though, so let’s change and make it slow down, instead of waiting 25 seconds, and changing lanes after it passes!
- Why should I get over a for a turn before the traffic bunches up at the intersection?
- Let’s change lanes for no reason!
- I’m just going to randomly signal!
This summer I did a 3000 mile road trip, and the vast majority of the highway miles were done with FSD version 11.4.4 doing the driving. It performed very well, and behaved as you would want.
Then the upgrade to 11.4.7, and later 11.4.7.2 and 11.4.7.3 came, and FSD has gotten much worse. It now so frequently tries to get in the wrong lane, or take wrong turns, that it’s too irritating (and potentially dangerous) to use. It will try to take freeway exits and will change into right turn only lanes on surface streets when the navigation route says go straight.
It always did that occasionally, but now it can happen every block on some streets, and potentially every exit on the freeway. This means instead of letting it drive, and occasionally overriding it, there’s just no point in turning it on.
Previously on my freeway commute I would engage FSD and let the car deal with it. Now I’m much more likely to just use cruise control, because at least that won’t incorrectly swerve into an exit lane 3 times in 10 miles.
I feel reasonably confident that full self driving cars will not exist within the lifetimes of anybody on this board, no matter how young they are. It’s not just the almost infinite availability of edge cases, and the difficulty of generalizing them, but also the massive amounts of capital it will continue to require.
Oh, and the need to write just a fuckton of code reflecting those edge cases that has to all work well together.
I don’t think “follow the navigation route” or “avoid turn lanes unless you intend to turn” are edge cases!
I’m not saying they’re easy to code, but they really are part of the core function of the system. Integrating map (stored lane information) and visual (signage and lane marking) data is probably difficult, but the correct decision should be easy when all of the data is in agreement.
The big flaw in 11.4.7.x seems to be some idea the software has that a different lane is always better, and particularly a new lane that appears must be really good.
As I’ve said many times, I do agree with the title of the thread.
What I mean is, they have code that’s working quite fine, but they have to keep introducing new code for the edge cases, which I suspect begins to conflict with existing code.
Having followed (and respected) your approach to, and descriptions of, FSD from the gitgo, I’d say that’s one hell of a regression. If they were paying any attention to your feedback, there’d be some headbanging at HQ.
For better or worse, FSD 11.x is dead. I expect that it’s in maintenance mode at best. And maintenance mode for software means regressions. 11.x has some AI-based components, but the core of it is a hard-coded rules-based system.
12.x is the new hotness. End-to-end AI. It won’t have the intricate set of rules that currently exist and (as said) undoubtedly are almost impossible to maintain, with every new edge case possibly causing a regression elsewhere.
It will, of course, have its own set of problems, and it’ll probably be worse than 11.x to start with. But just as LLMs have proven to be vastly more flexible than expert systems, I think ultimately we’ll see something similar here. And Tesla has the training data to make it happen.
Trying to build full self driving with a rules based system was always destined to fail. There are just too many permutations and edge cases. The real world is too complex to model.
However, the new tack of putting essentially an LLM into the car makes a lot of sense. It doesn’t have to be a full AGI (although that might be close), but it needs to have AGI-like decision-making within the driving environment.
Quality assurance will be a bitch. You can’t test code you don’t understand and didn’t write, and which is encoded across the weightings of billions of parameters.
What will likely happen is that LLMs will be ‘trained’ by existing data, by watching as humans drive, etc. Perhaps use sensors and cameras to record the road as a human drives, and have the AI record what it would have done at each moment, then have that scored through RLHF or automatically by comparing its choices with the human’s. Eventually, the AIs will make the same kinds of decisions people make when faced with novel situations.
I used to think full self driving was really decades away. I’ve been saying so for… decades. But my mind is changing with the new AIs. I think we might see competent AI driving at least as good as human driving within five years.
One thing they’ve talked about, with regards to the training set, is figuring out which are the good drivers. They have millions of vehicles on the road that are feeding back data, but really they only want the training data from the best of them. I wonder if they’ll have to train another AI just to figure out the difference between good and bad driving.
Negative reinforcement might also be a possibility. They can find scenarios where a mistake was made and feed those back to it with negative weight.
It’s not just me, either. Electrek reported the same thing soon after it came out. Other people on forums have encountered similar.
Tesla FSD Beta tried to kill me last night | Electrek.
11 replaced freeway rules based with AI, and some things improved, but others did not. I’m generally fine with software changing, but going on 2 months of even the old style enhanced autopilot being useless is frustrating.
I’ve been using open source and free software for over 30 years, so I’m very familiar with the extremely annoying practice of perpetual alpha. They start working on some release, but just as it’s getting stable, and needs some attention to cleanup bugs, the developers all shift to working on the next major version, which is more interesting. Leaving the users with an old version full of known problems, or a new version not yet ready for use.
It’s frustrating enough when that is some Android fork being done by a few guys, but pretty unacceptable when it’s software people payed thousands for.
One feature of 11 (or maybe later) was accepting a voice memo about why FSD was canceled. To be done at scale you’d need something to classify all of the notes into a negative training set.
One funny thing that came out of the FSD 12 training data: humans only come to a complete stop at stop signs <0.5% of the time. Since the car mimics what humans would do, it also doesn’t come to a complete stop all the time.
Is that a problem? Well, it’s illegal, and it seems like it’s a bad thing if the default behavior of a car breaks the law. On the other hand, it’s not much of a law if it’s broken 99.5% of the time by humans. Worse, if other humans on the road expect that behavior, then the car is possibly causing an unsafe condition if it doesn’t do that.
Well, it’s a bit more than the next version being more interesting. It’s an open-ended research project. No one knows yet, even in principle, what the right architecture is. Every past major version of FSD/Autopilot has seen rapid initial gains in capability, and then it slows or even reverses a bit as time goes on. They take some lessons about how to rearchitect things and launch the next version, which is often a bit worse initially but reaches a higher peak capability. Rinse and repeat.
I don’t see any way around it. Yeah, it’s unacceptable in traditional software terms, but it’s not like building an OS or a web browser, where we already know from past examples how to put things together.
Also, FSD 12 couldn’t have come out fully formed. Previous versions were required to bootstrap it into basic capability. It’s at the point now where it can learn purely by feeding it more videos, but that wasn’t true initially.
I use this pretty frequently, but the note is always “it tried to change lanes when I didn’t want it to.” I wish the option for less-aggressive lane changing was sticky and not per-drive.
People report their cars uploading gigabytes of data to Tesla, so I’m sure they’re keeping all of it for their training set.
They also bought a shitload of top-end GPUs from my company, which makes me happy 
.
Math error or typo: that is only 7,300 hours of experience.
Typo, yeah, that’s it 
Other than some early DARPA projects, I don’t think anyone was really trying to do this, though. The shift, I think, has been from building a collection of DeepMind-like AIs to a more generalized, single AI.
If we go up in this thread we can probably find people criticizing Tesla’s decision to ditch LIDAR. It seems insane to handicap your ML efforts by reducing inputs. However, at this stage, it seems like that was absolutely the correct move. If you look at AIs like AlphaStar, they achieve their dominance (in Starcraft II, in this case) by doing a little bit of cheating. AlphaStar gets all of its information directly from the game engine, so it doesn’t have to “look around” the map to see what’s going on. It doesn’t have to process visual information to determine attack patterns or build times or resource allocation. It can also make inputs directly into the game engine at crazy speeds.
The next version of AlphaStar would impress only if it had to use a keyboard, mouse, monitor, and speakers, and play the game with the same basic limitations that a human player has.
Likewise, the shift I’ve seen in autonomous vehicles over the last 18 months (as a very casual observer, so take this with a big grain of salt), is away from specialized inputs and handlers and instead relying on stereoscopic cameras learning to drive from scratch the way a teenager does. This means a couple of things, I think. One is that any money companies have sunk into R&D up until now has essentially been wasted. Sure, there’s the body of knowledge that’s been produced, but ChatGPT and the like have shown us that a startup can basically come in at some point with a similar approach and catch right up to the major players.
The other is that, while I still think self-driving cars are decades away, I think when we finally get there we’ll be about 6 months away from a humanoid robot being able to get into any vehicle and drive around using standard human controls and 2 stereoscopic cameras where eyeballs go. That’s the end state, as I see it. Once an autonomous vehicle is relying solely on the same information that a human driver has, only then will it be able to share the road with human drivers and not be weird. And by the time we get to that point, we’re probably 3 years away from the singularity anyway.
I mean, just a reminder here that Tesla loudly announced that they were done with the radar, but have been very quietly reinstalling a new version it. (no, I don’t expect them to go back to LIDAR)
With respect to generalized AI, can folks tell me the actual mechanism for how that might happen? Will there be one massive AI brain somewhere constantly learning, which will then download its learning updates to each vehicle? I don’t know enough about it to have a clear picture.
I agree mostly with the OP.
To me, the biggest drawback of CAVs is. . . why? Why do we need them? We don’t. That coupled with technological limitations makes them a mostly a low priority.
If you look back at other technologies, they provided amazing improvements, phone, e-mail, Internet, TV. . . but, a CAV is not a big improvement over what we have. My commute has gotten easier over the years, not more difficult, why do I need a CAV?
I work in transportation, and at a conference recently a local area rep talked about the CAVs they were using to shuttle passengers back and forth from a parking lot. Pretty cool. Until they mentioned that the CAV would get confused if the grass on the side of the route got too high, so they had to adjust the mowing schedule.
Wait, what does CAV mean?
Self drive vehicles that actually work would be an enormous improvement over letting people drive. Colossal improvement. We currently have over 40,000 deaths a year due to people driving, that can be a bit better, I think.