I think it’s hard to implement guardrails for AI because ultimately it just optimizes a solution for a given problem and just sort of views everything as “data” without assigning it any sort of inherent morality, so to speak.
That’s the greatest danger I think. Not that it will become this monolithic self aware entity, announce itself to the world, and send a bunch of Terminators to kill everyone. More like we build this self-perpetuating “system”, by which I mean our increasingly networked and data-driven civilization, into an automated feedback loop that seeks to maximize for human needs and wants, continuously trained on data received from the humans it interacts with.
I mentioned this in another thread, but think of ants or other insects with a hive mind. They don’t understand the big picture. They just receive chemical messages to “find food” or “kill rival ant colony”.
Well I sort of feel like our current system is a bit like that. We are all connected to this feedback loop that seems to send messages via algorithmically generated social media messaging “work harder”, “cut off social ties”, “people who work on AI are receiving wealth beyond your wildest dreams”, “here is some porn and inane short-form videos to placate you if you have nothing AI-related to work on”.
Sort of like “They Live”, but with more overt messaging.
This was actually the plot of “Goodbye, Good Luck, Don’t Die”. The protagonist ostensibly time traveled from a future where society collapsed because the AI basically evolved to give people want they wanted…sit and experience VR all day.
The actual mechanism for societal collapse (maybe not “end of humanity” level) I think would be the myopic pursuit of throwing critical resources and infrastructure at the pursuit of some “AI God” that we believe will solve everything. The end result being massive unemployment, inflation, wealth disparity, etc. Sort of like the way Jared Diamond describes the residents of Easter Island burning all their resources to build those stupid Moai heads.
Or something simply goes wrong with it and we no longer have humans who can actually do the work manually. That was a scenario I read recently in The Atlantic I think.
Or maybe AI will be nice and helpful like Samantha from Her. Who knows.