On this part…
As a computer programmer, I think some foundational “epistemology of machine learning” kind of introduction would be helpful, if only to better differentiate between the deterministic outputs of traditional computing vs the probabilistic outputs of LLMs. It’s a pretty fundamental change in the way we use computers as a tool, and has not only practical applications (how to prompt, modify temperature, fine tune, minimize hallucinations, etc.) but also philosophical and ethical ones (not just “is it conscious” but more fundamental concerns like how does it acquire new knowledge, how it approaches things it doesn’t know, how its memories and contexts work), etc.
You need quite a lot of math and software to actually be able to produce such a model, but a layman’s overview of the basic flow doesn’t need to be advanced and would still do a lot of good, IMO. Similar to how you don’t really need to be a rocket scientist and able to calculate trajectories just to understand that objects in space orbit each other and cause all sorts of earthly phenomena from tides to seasons.
I don’t think “how to make AI” should be a required part of the curricula. For that matter I didn’t agree with the idea of computer science/programming should be taught to everyone, either. It’s just another trade skill and absolutely isn’t right for everyone. It would be a hugely boring waste of time for many people.
I’d rather we teach more humanities and things like civics and home life and basic finance and scientific literacy, instead, all of which schools aren’t great at.
Having a bunch of unemployed engineers doesn’t really prepare our society for the future, it just causes a race to the bottom in the labor market as technology and capital continue to consolidate. Having good citizens actually able to steer society and government, on the other hand, might better prepare us for steering the ship as needed.
Back in the real world, though, we’d probably get neither, and just end up with the poor librarian teaching “how to AI” in the 20 minutes after “how to Google”. Our educational system is optimized for spitting out elites at the top, not so much bringing up the bottom at the other end.