# Crowdstrike failure - How? Why? **URL:** \ **Category:** Factual Questions\ **Created:** [July 20, 2024, 1:54pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492 "2024-07-20T13:54:58Z")\ **Posts on this page:** 20\ **Page:** 6
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [July 26, 2024, 10:29am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/101 "2024-07-26T10:29:47Z")
Why don’t you just get a normal setup, and make sure it never goes wrong. ---
**Author:** ![Cervaise](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/cervaise/32/16693_2.png) [@Cervaise](https://boards.straightdope.com/u/Cervaise)\ **Post date:** [July 26, 2024, 10:33am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/102 "2024-07-26T10:33:10Z")
Yeah, one of the big, long-standing takeaways for lay observers asking about these sorts of tech failures has to be — there’s the right way to do a thing, and then there’s the way the thing actually gets done when management finds out how much the right way costs. ---
**Author:** ![FinsToTheLeft](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/finstotheleft/32/2930_2.png) [@FinsToTheLeft](https://boards.straightdope.com/u/FinsToTheLeft)\ **Post date:** [July 26, 2024, 10:50am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/103 "2024-07-26T10:50:00Z")
> [@Mangetout](#): > > Why don’t you just get a normal setup, and make sure it never goes wrong. A friend once said that with a sufficiently powerful CPU we could do everything in Excel. The reality is that we need to deal with patching, hardware maintenance, power issues, etc. and there is a cost to this. Accepting an outage of 60 minutes is often the alternative. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [July 26, 2024, 11:15am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/104 "2024-07-26T11:15:07Z")
‘Accepting’ the risks was a thing I had fun with in my last IT role; typical options for risk managment are: - Avoid (do something in advance so the risk simply can’t happen - like, dispose of the risky system) - Mitigate (do something to make it less likely to happen, or less severe if it does happen) - Transfer (give it to a third party to manage) - Accept (do nothing, just understand that it will be a bad day if it does happen) But most of what I kept getting told by senior management was: “We accept this risk, so there’s no need for you to waste time and money mitigating it. Just make sure it doesn’t happen” ---
**Author:** ![Cervaise](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/cervaise/32/16693_2.png) [@Cervaise](https://boards.straightdope.com/u/Cervaise)\ **Post date:** [July 26, 2024, 11:23am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/105 "2024-07-26T11:23:49Z")
Heh. I recently took over my company’s risk-assessment function, and when I revised the board’s presentation template, I made sure to include a definition section whose first entry said that accepting a risk meant no special mitigations were in place and no further action was planned. They didn’t like that. Too bad. ---
**Author:** ![FinsToTheLeft](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/finstotheleft/32/2930_2.png) [@FinsToTheLeft](https://boards.straightdope.com/u/FinsToTheLeft)\ **Post date:** [July 26, 2024, 11:26am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/106 "2024-07-26T11:26:22Z")
> [@Mangetout](#): > > But most of what I kept getting told by senior management was: “We accept this risk, so there’s no need for you to waste time and money mitigating it. Just make sure it doesn’t happen” “Thoughts and prayers!” ---
**Author:** ![LSLGuy](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/lslguy/32/5813_2.png) [@LSLGuy](https://boards.straightdope.com/u/LSLGuy)\ **Post date:** [July 26, 2024, 1:53pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/107 "2024-07-26T13:53:34Z")
If most sales types and C-suite people didn’t have magical thinking, they’d have no thinking at all. ---
**Author:** ![Northern\_Piper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/northern_piper/32/5304_2.png) [@Northern\_Piper](https://boards.straightdope.com/u/Northern_Piper)\ **Post date:** [July 30, 2024, 12:16pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/108 "2024-07-30T12:16:15Z")
And Crowdstrike enters the next phase: potential litigation. Delta Airlines is reported to have hired David Boies for a potential lawsuit. Delta’s losses are estimated at around $500 million. > **[Delta lost a ton of money because of CrowdStrike. It wants some back.](https://www.businessinsider.com/delta-hires-david-boies-lawyers-crowdstrike-microsoft-pay-damages-2024-7?amp)** > > The carrier has hired star attorney David Boies to seek damages from CrowdStrike and Microsoft for the computer outage, CNBC reported. Globally, the cost of the Crowdstrike matter is estimated at around $5 billion. > **[We finally know what caused the global tech outage - and how much it cost |...](https://www.cnn.com/2024/07/24/tech/crowdstrike-outage-cost-cause/index.html)** > > Insurers have begun calculating the financial damage caused by last week’s devastating CrowdStrike software glitch that crashed computers, canceled flights and disrupted hospitals all around the globe — and the picture isn’t pretty. ---
**Author:** ![Francis\_Vaughan](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/francis_vaughan/32/3093_2.png) [@Francis\_Vaughan](https://boards.straightdope.com/u/Francis_Vaughan)\ **Post date:** [July 30, 2024, 1:28pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/109 "2024-07-30T13:28:36Z")
As of today, Crowdstrike’s market cap is 68 billion. Given they have shed well over 10% of value since the disaster, they have lost a similar or greater amount themselves. Which is somewhat ironic. Maybe the best answer is for them to negotiate a settlement paid in shares. No matter what, getting past the looming legal storm early would probably boost their share value more than fighting ever would. But wit ould take a CEO with balls of tungsten carbide. ---
**Author:** ![Northern\_Piper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/northern_piper/32/5304_2.png) [@Northern\_Piper](https://boards.straightdope.com/u/Northern_Piper)\ **Post date:** [July 30, 2024, 1:33pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/110 "2024-07-30T13:33:01Z")
And I know that Crowdstrike no doubt has iron-clad disclaimers of liability built-in to every licensing agreement, as the article mentions. But when the magnitude of the loss is so high, and the cause of the loss is such an obvious error, possibly amounting to gross negligence (the “no testing needed, we’ll just fix it if it breaks something” attitude, mentioned earlier in the thread, if proven), I could see the lawsuits getting traction. Plus, if it’s not just Delta, but a stream of lawsuits from others, keeping Crowdstrike’s negligence in the news for a few years, that’s not good for their market reputation. ---
**Author:** ![Railer13](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/railer13/32/9742_2.png) [@Railer13](https://boards.straightdope.com/u/Railer13)\ **Post date:** [July 30, 2024, 5:35pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/111 "2024-07-30T17:35:23Z")
From the linked article: > Boies has represented Theranos founder Elizabeth Holmes, Al Gore in the 2000 presidential election, and the US government in an antitrust case against Microsoft in 1998. Not a stellar track record, IMO. ---
**Author:** ![Northern\_Piper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/northern_piper/32/5304_2.png) [@Northern\_Piper](https://boards.straightdope.com/u/Northern_Piper)\ **Post date:** [August 7, 2024, 4:29am UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/112 "2024-08-07T04:29:40Z")
Microsoft has found someone else to blame for Delta’s problems. : Delta! Says Delta’s outdated software made the Crowdstrike problem even harder to fix. Delta denies it, and says that Crowdstrike was nowhere to be found when they needed help. MS says they offered help, but Delta declined it. > **[Microsoft lashes out at Delta: Your ancient tech caused the service meltdown...](https://finance.yahoo.com/news/microsoft-lashes-delta-ancient-tech-183813752.html)** > > Microsoft on Tuesday blamed Delta Air Lines for its weeklong bout of cancelations last month, claiming its aging technology caused the airline to recover far slower from the global tech outage than its rivals. Microsoft also claimed Delta turned down... ---
**Author:** ![Northern\_Piper](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/northern_piper/32/5304_2.png) [@Northern\_Piper](https://boards.straightdope.com/u/Northern_Piper)\ **Post date:** [August 8, 2024, 12:16pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/113 "2024-08-08T12:16:29Z")
And the litigation continues to mount: Delta the target of a class action by passengers: > **[Delta passengers sue airline over refund refusals after CrowdStrike meltdown...](https://www.cnn.com/2024/08/07/business/delta-passengers-sue-crowdstrike-meltdown/index.html)** > > Some Delta Air Lines passengers whose flights were canceled when the airline melted down due to a global tech outage in July have sued the airline in a class action. ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [August 8, 2024, 3:04pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/114 "2024-08-08T15:04:07Z")
> [@HMS\_Irruncible](#): > > It’s possible Crowdstrike didn’t test correctly, but it’s notoriously hard in software to test every single permutation of every configuration of every customer. As a vendor I cannot possibly do this, but customers know their specific setups well enough to keep a test lab maintained. That is true. In my data structures class a few of us tried to make our programs fancy and then when the professor would run his data through and there would be a zero somewhere that would crash our menu system or error catching (ironic) or something. But in this case, all nulls? No way that passes even alpha testing. ---
**Author:** ![Saint\_Cad](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/saint_cad/32/18907_2.png) [@Saint\_Cad](https://boards.straightdope.com/u/Saint_Cad)\ **Post date:** [August 8, 2024, 3:08pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/115 "2024-08-08T15:08:19Z")
> [@Northern\_Piper](#): > > And the litigation continues to mount: Delta the target of a class action by passengers: As a person that showed up to DEN at 6:00am on that morning for a flight to Rome, I can tell you that some airlines were able to adapt (OMG! Actually going to give Air Canada props on this one. 😲) Others had the attitude of there’s not a thing I can do about it. I wonder if that will enter into the liability somehow. ---
**Author:** ![BigT](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/bigt/32/12044_2.png) [@BigT](https://boards.straightdope.com/u/BigT)\ **Post date:** [August 8, 2024, 3:23pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/116 "2024-08-08T15:23:02Z")
I did just learn recently that the parser is actually Regex based! For those who don’t know, Regex is short for Regular Expressions, and is a syntax used primarily for searching inside strings of text. It also commonly will allow you to basically do “Find and Replace.” One common flaw in users who first learn of it is to try to use it to do everything. For example, /a.[0-9]/ would match the letter a, followed by one other character, and then a numeral. So it would match “ab2” in the string “bab2cat”, “a32” in the string “x0oa32” but would not match anything in “a5od2”. I’m not all that knowledgeable in this sort of programming, but using Regex as your parser for important files does sound alarm bells in my head. For one thing, Regexes don’t return errors. ---
**Author:** ![gnoitall](https://avatars.discourse-cdn.com/v4/letter/g/bb73d2/32.png) [@gnoitall](https://boards.straightdope.com/u/gnoitall)\ **Post date:** [August 8, 2024, 3:38pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/117 "2024-08-08T15:38:04Z")
> [@BigT](#): > > One common flaw in users who first learn of it is to try to use it to do everything > **[Perl Problems](https://xkcd.com/1171/)** > > To generate #1 albums, 'jay --help' recommends the -z flag. ---
**Author:** ![FinsToTheLeft](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/finstotheleft/32/2930_2.png) [@FinsToTheLeft](https://boards.straightdope.com/u/FinsToTheLeft)\ **Post date:** [August 8, 2024, 5:03pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/118 "2024-08-08T17:03:06Z")
> [@Saint\_Cad](#): > > OMG! Actually going to give Air Canada props on this one. 😲 I think Porter was the only Canadian airline hit by the Crowdstrike FUBAR. ---
**Author:** ![markn\_1](https://avatars.discourse-cdn.com/v4/letter/m/f9ae1b/32.png) [@markn\_1](https://boards.straightdope.com/u/markn_1)\ **Post date:** [August 8, 2024, 5:13pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/119 "2024-08-08T17:13:54Z")
> [@BigT](#): > > I’m not all that knowledgeable in this sort of programming, but using Regex as your parser for important files does sound alarm bells in my head. For one thing, Regexes don’t return errors. I’m not sure what you mean by that. A regex match will certainly tell you if the string doesn’t match the pattern. That’s the whole point of a regex match. Regex has limitations in the type of pattern it can match, but if the pattern you’re trying to match is of an appropriate type, using regex is perfectly fine. ---
**Author:** ![HMS\_Irruncible](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/hms_irruncible/32/7394_2.png) [@HMS\_Irruncible](https://boards.straightdope.com/u/HMS_Irruncible)\ **Post date:** [August 8, 2024, 6:30pm UTC](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492/120 "2024-08-08T18:30:45Z")
> [@Saint\_Cad](#): > > That is true. In my data structures class a few of us tried to make our programs fancy and then when the professor would run his data through and there would be a zero somewhere that would crash our menu system or error catching (ironic) or something. But in this case, all nulls? No way that passes even alpha testing. I suspect this was a packaging/distribution issue. A lot of software will go through exhaustive testing of the business logic because that’s what is most likely to cause catastrophic failure. But people will cut corners on testing the packaging and distribution because this rarely changes, it adds cost to the test cycle, and it’s uncommon for bugs to show up there. [Previous page](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492.md?page=5) [Next page](https://boards.straightdope.com/t/crowdstrike-failure-how-why/1004492.md?page=7)