Too late. As much as $5 billion drained out of user’s wallets.
Crypto traders had a rough Thursday. Within hours of each other, two of the biggest names in the space reported security incidents: Polymarket, which lost user funds for the second time this year, and Binance, which watched one of the largest outflows in crypto history leave its wallets in under an hour. Security researchers have already been calling this month “Black October” after a wave of AI-assisted vulnerability discoveries and breaches, and these incidents land right in the middle of it.
I was actually thinking of security vulnerabilities discovered in SWIFT or other parts of the actual world financial system. You know, once some shady company managed to make an unauthorized charge against my checking account. I wrote a letter to the bank, and, lo and behold, my “drained” funds were restored!
Imagine the Binance and Polymarket hacks were just ordinary heists. Imagine a hack that disappears trillions of dollars. That’s gonna leave a mark on world banking and markets.
Be a little tough to do that given 90% of banking still runs off of Excel and Powerpoint.
But assume whatever AI technology is being used for hacking, infosec people are making money selling AI based security software to counter it. It’s a bit of an arms race, same as it has always been.
That said, I still wouldn’t trust linking ChatGPT to any of my bank accounts. Mostly because I have no idea what data privacy protections are in place or what safeguards prevent me from accidently writing a prompt to short pumpkin futures on Nov 1.
If a bank has no backup records so that trillions can simply “disappear” even due to malicious activity, yeah, I am going to take my business elsewhere.
Why do I suddenly remember rogue traders wiping out Barings Bank and others? At least an AI can do that better and faster
This is almost inevitable. A system wide brain, or a Matrioshka brain occupying most of a Dyson swarm, would have issues of signal latency; one side of a swarm orbiting a star at 1 AU would be separated from the other side by sixteen light minutes.
It would take at least half an hour for the entire swarm to reach a consensus over whether to say ‘Hello’ or not.
AI doesn’t act maliciously or feel greed or play politics or double down on “gut instinct” or get coked up and spend the weekend at wherever the equivalent of The Hamptons is for City bankers. Not that there haven’t been issues of automated trading systems running amok. But generally, the reason banks like Barings, Lehman, Bear Stearns, SVB, and others failed is that humans made bad business decisions or circumvented controls (or failed to implement them in the first place).
One might argue that there is almost no need for humans on Wall Street. In theory, AI should be able to ingest market data, corporate financials, the weather, the price ot paperclips, pretty much every bit of data that could influence a stock’s price and come up with a proper valuation of a stock.
Of course the reason it can’t is because all that data doesn’t just sit in some huge public data lake somewhere. So there is still an imbalance of information.
One thing that’ll be interesting and terrifying for a lot of people is when AI accomplished goals, but we can’t understand the chain of reason that leads to it.
It would be like if humans tried to help squirrels achieve their goals. Squirrels like to eat various kinds of nuts, so humans would have to explain to them that step one is the humans need to go to the other side of the planet and dig some holes. The squirrel won’t even comprehend concepts like ‘the other side of the world’ and it won’t understand why digging holes helps it achieve its goal of eating more nuts. But the humans need to go to the other side of the world to mine for fossil fuels, metals, etc so they can build the industry to support agriculture, which will result in more nuts. Squirrels can’t comprehend this chain of logic of how step 1 of digging for holes on the other side of the world results in step 11, increased nut production, and will find it confusing.
I’m guessing in the 2030s the same thing will happen. We ask an AI to help us solve MS and its first step is learning to play the trombone for Mardi Gras in New Orleans. People will get upset and say its wasting its time, but cause and effect will eventually lead to the advances needed to cure MS in ways that don’t make sense to us.
The point is people will be both terrified and in awe when AI achieves these things and even when it tries to explain it to us, our brains can’t comprehend all the moving parts and cause/effect that led to the desired result.
After writing that I realize that that I just the plot of the movie ‘everything everywhere all at once’. But the point still stands. We will see the chain of reasoning and we won’t be able to follow it, we will only see the results and that will terrify a lot of people since it makes them realize they aren’t in control anymore.
I think there’s a tendency to anthropomorphize AI or think of it as omniscient because it can process so much data and interact in a natural language way. And our society already operates in a way where most people don’t understand all the moving parts. We just take it for granted that food shows up on the supermarket shelves and the trains show up on time(ish).
I think once we get over the initial awe of AI and realize it’s not magic and can’t really replace every single task humans do, we’ll reach a new equilibrium where it’s just another service that sits in the background.
