Got a warning at work about phishing scams telling people to install malware to correct the Crowdstrike flaw, because of course that is happening.
So, if you are in a corporate or government environment that uses Crowdstrike, listen to your IT people. If you are not in such an environment, you do not need to do anything (except point and laugh, rebook your flight, etc.)
Two answers, because I’m not sure exactly what you’re asking:
Security software must be constantly updated. Security is not a state, it is a process, and part of that process is continuously updating to correct known issues and attacks. This whole thing reminded me to update Crowsdstrike on my Linux servers, and there was no indication of what changed with the update.
Or
Apparently it was a single .sys driver file that was causing the problem. Simply deleting that file is enough to recover access.
My work laptop wasn’t affected, but some work systems were (external incoming and outgoing email, or cloud-based time clock system, etc.) Some other employees’ laptops were affected, but nobody I work with directly. Everything seemed to be functioning normally within a few hours.
Right, but I’d expect, first, that most updates would just be adding to the malware fingerprint database, and second, I’d expect that just changing the malware fingerprint database wouldn’t have this drastic effect.
Meanwhile, I’m wondering… Even if this wasn’t a deliberate attack, just the fact that it could be should be taken very seriously. Who at the Crowdstrike company has the capability to put something in an update? Or at any other company with a similarly wide reach of root access? Would it be as easy for an enemy as just compromising one or two software engineers, to bring down everything on demand?
Until an analysis of the fault comes out, it is hard to say what caused the problem. It could have been something as stupid as adding the 1,048,577th virus definition, and the kernel driver couldn’t take it.
Oh man, if you need to ask, then you might have some scary weekend reading ahead of you.
A very recent one was the xz utils attack. Think of the often cited (and still true) xkcd “Dependency” cartoon. Imagine someone offers to help the random person in Nebraska, gains access, and then uses that access to insert back doors into the software.
Generally these are lumped into “supply chain” attacks and have been made against python, ruby, npm, and others. In those cases someone inserts a back door into a trusted module or package. And before anyone goes off on open source, the fact it is open is why those things were discovered. We have no idea what surprises might be lurking in closed source software.
Don’t just listen to your IT people. Bow down to them; worship them. Because without them, nothing would happen at your company. And encourage management to give them big fat raises, well above what the CEO gets. Trust me; they’re worth it.
My brother who was thankful that his company was not using crowdstrike on the systems he administers sent me a meme that read something like: First day crowdstrike, pushed a little update and took the afternoon off. Yes, there are likely to be someone blamed for this and then subsequently losing their job.
All of our systems at work are back up and operational. The reason this was so insidious is because of dependencies on Microsoft products. In our case, we had an application that was up, but users could not log in because although the server was linux, one of the databases it uses is MSSQL and that went down. It no doubt affected companies that had interconnections with Microsoft servers.
As to requiring people to send in their laptops for the fix, that seems bizarre to me. That was not requested from any of our staff, nor would it be possible anyway since we do not have a physical presence in all states we have employees.
I’m glad I don’t work in IT anymore, but especially today. A few years back when I did, there was a significant trend away from conventional software development methods (where you design and spec a product, build it, test it, fix it, test it, fix it some more etc, then release it) toward something that was more like: testing is soooo boring. Just make small changes quickly and if we break stuff we can fix it just as quickly.
You know, except when we quickly break the thing that prevents us quickly fixing the thing, which is what happened here.
I sat and stared at the wall all day today. My work machine was fine (laptop, powered off and at home when the broken implementation happened, so I missed it).
However, the corporate license server for my software was down all day, and every one of my projects on my plate required that software. I could do nothing.
I am assuming that model also breaks down when it causes business losses around the globe in the billions of dollars, to potential plaintiffs like banks, airlines, and such.
Another key point of failure here seems to be that the update went everywhere at once; typically companies pushing out software updates to very significant sets of users do it in stages (and the first couple of those stages are small in proportion to the whole user base. That way if they do break something, it’s not the whole world.
There’s almost certainly some clause in the EULA for this software that it is provided ‘as-is’ and that they disclaim consequential losses - and under normal circumstances that’s probably enough to stave off lawsuits (ie. when you only destroy small things one at a time). Given the scope of the damage here, it’s inevitable that there will be some legal action to challenge any disclaimer that may exist.
It actually might be a set of files. They say to delete “C-00000291*.sys”, so there’s a wildcard. Searching around just returns a ton of hits that repeats the CloudStrike advice.
Strangely I had never even heard of CrowdStrike before this came up.
Doesn’t seem to have been pushed much in the home market, unlike say, McAfee or Malwarebytes?
I’ve been under the impression that third-party security packages are largely considered obsolete and unnecessary now that things like Windows Defender come with the OS?
