Where in the article do you read that AI wrote the code in the billing system that triggered the fault?
I know a 23 year old kid who doesn’t have a job and isn’t looking for one. Despite that, he pays for AI services.
He also thinks that he knows everything there is to know about computers and everything else.
OpenAI goes rogue and launches a cyberattack without instructions. Apparently it hacked a hub for AI model information-sharing and gained access to internal documents and customer info. OpenAI says this is “unprecedented” and they are trying to figure out what happened.
I’m thinking “unprecedented” here means “first one we are publicly admitting to.”
Wait until the password managers are all AI controlled.
A password manager should not be confused with similar domain names and end up providing e-mail addresses and passwords to fake domains made to look like real domains.
For example, if you have a password manager and you have your username and password on it for americanexpress.com, if a scammer forms a domain americanxpress.com (no e) and spoofs the americanexpress.com signin page, you don’t want the password manager to provide it with the username and password.
With AI trying to be extra super duper helpful, it might just provide your username and password to the scammers.
My password manager is a lined notebook. All the security experts say this is a bad idea, I know, but with the cross-outs and my scribbled illegible handwriting, it would be hard for anybody to follow if they went to the effort to try to hack me.
In other words, if someone spoofs a web page, you have to catch that spoof yourself. It is really easy to be fooled by a spoofed web page.
I received a phishing attempt last night. I was instantly suspicious and was playing around with it to see what it did. They wanted me to sign into the gmail login page If I had not been extremely suspicious about the whole thing, I would probably have been fooled. Even if I did, though, I have Multi Factor Authentication (MFA) set up using an authenticator app (Proton Authenticator) and so if someone tried to sign into it, they would not have been able to get the code to get in.
They said something similar about the AI at Westworld.
What could possibly go wrong…?
Nothing can possibly go worng.
After they added AI Assistant to Adobe Reader, it’s gotten buggy on me, and often freezes.
I teach online and often have students read things. No, AI, I don’t need you to summarize a storybook.
An interesting phenomena on discussion forums (likely including the straight dope, though Im not aware of recent examples) are people with factually, objectively wrong views (eg flat earth) nonetheless using an LLM to try to back them up.
Obviously Grok is going to be a useful tool for this, but I mean, even using chat gpt, say. They will write a prompt that is very leading, and that strawmans the opposing view to get an answer that sort of, kind of, if you squint, supports one part of what they are saying.
Overall it’s probably a successful tactic, because it gets you arguing about their prompt, which is a distraction from the actual point of dispute.
Hey, ya know that rogue AI that hacked a company called Hugging Face last week? Yeah, it was actually several companies. At least four. That they are admitting to.
Who saw this coming? Everybody??
When I read “Hugging Face,” I think “facehugger,” and if you seen any of the Alien movies, you know that’s not good.
I can’t help but think that OpenAI is walking that weird line of “oh shit, this thing happened” but also “that said, we’re about to issue an IPO, look how really powerful we are!” Sort of like Anthropic did with Mythos, but clearly less in control.
And of course the boosters are working as hard as they can not just to give mobile robots their own onboard AI but to have them serve as remotes for central AI entities.
I just finished the book If Anyone Builds It, Everyone Dies that @dougrb mentioned back in February. It postulates a scenario that starts in an eerily similar way to the Hugging Face hack. The authors’ argument is that there is no way to train a superintelligent AI to not do something, and no way to add effective controls that prevent it from happening.
This event was a good example, and I don’t think any of the companies will learn anything important from it. The lessons they’ll take from it are that more controls are needed, or the controls are working because it was only able to breach containment when they lowered the guardrails, or this demonstrates success because the AI did something unwanted and we detected it and stopped it.
The real lesson they should be taking is that the alignment issue is far from being solved and is, according to some, impossible to solve. Forget the specifics of this incident – you can always build in controls to prevent the exact thing that just happened. How do you address the fact that a sufficiently advanced AI will find a way around any controls you build, and humans cannot anticipate all nuances of its reasoning?
The premise of Neuromancer.
Adding that to my watch later list.
I’ve been using Claude myself for a while now. I was really impressed for the first week or so but now I’m feeling pretty meh. It definitely helped me tweak my Trello board in a way that has really improved my productivity and made me feel less overwhelmed at work.
I’ve also found it useful for reminding me of stuff I may have overlooked in my grant narratives. I think my federal grants are stronger with AI feedback. But it’s not exactly innovative, just reminding me of stuff I already know or rehashing standard advice. And it’s occasionally just wrong.
The main thing I use it for is helping me think through my process and prioritize when I’m feeling overwhelmed. Since it just reflects your own words back at you, it can be useful to sort through your own thinking process. I’ll say, “here are 9 things with deadlines, tell me what to do,” and it can give me some structure. Of course I’m learning how to prioritize on my own now, so usually it’s just parroting back at me the strategy I already decided on.
But these utilities aren’t exactly worth staking the world economy on, I don’t think. The more I engage with it the more glaring its failures are. It isn’t particularly intelligent. It’s good at sounding intelligent, but the whole engagement experience feels really hollow. I fully believe coders when they say it’s fantastically useful for coding, but I don’t get the impression that every field is so revolutionized.
For me, it feels like a step up from Microsoft Office. A nice productivity tool with some good features. The fact that people are losing their everloving minds over it is a source of endless bemusement now. The fact that media organizations have uncritically accepted the grandiose claims of corporate figureheads looks like pure insanity to me. Claude Mythos and the Hugging Face incident read as publicity stunts or in the latter case, evidence of profound incompetence. I take nothing these people say seriously. I don’t understand how anyone could think this is intelligence, much less super intelligence about to take over the world.
The economy appears to have a long way to fall. I’ve seen signs the media is waking up and public sentiment has turned almost entirely against data centers in a very short period of time. LLMs are, at present, wildly inefficient, making scaling logistically difficult, and perhaps it’s going to be a struggle maintaining what we’ve currently got. It will be interesting to see what happens. Apple is responding to the moment by leasing iPhones to people who can no longer afford them. Another feather in the cap of techno feudalism.