CrowdStrike effectively bricked windows, Mac and Linux today.
Windows machines won’t boot, and Mac and Linux work is abandoned because all their users are on twitter making memes.
Incredible work.
this post was submitted on 19 Jul 2024
2029 points (99.1% liked)
linuxmemes
21575 readers
503 users here now
Hint: :q!
Sister communities:
Community rules (click to expand)
1. Follow the site-wide rules
- Instance-wide TOS: https://legal.lemmy.world/tos/
- Lemmy code of conduct: https://join-lemmy.org/docs/code_of_conduct.html
2. Be civil
- Understand the difference between a joke and an insult.
- Do not harrass or attack members of the community for any reason.
- Leave remarks of "peasantry" to the PCMR community. If you dislike an OS/service/application, attack the thing you dislike, not the individuals who use it. Some people may not have a choice.
- Bigotry will not be tolerated.
- These rules are somewhat loosened when the subject is a public figure. Still, do not attack their person or incite harrassment.
3. Post Linux-related content
- Including Unix and BSD.
- Non-Linux content is acceptable as long as it makes a reference to Linux. For example, the poorly made mockery of
sudoin Windows. - No porn. Even if you watch it on a Linux machine.
4. No recent reposts
- Everybody uses Arch btw, can't quit Vim, and wants to interject for a moment. You can stop now.
Please report posts and comments that break these rules!
Important: never execute code or follow advice that you don't understand or can't verify, especially here. The word of the day is credibility. This is a meme community -- even the most helpful comments might just be shitposts that can damage your system. Be aware, be smart, don't fork-bomb your computer.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Is there a good eli5 on what crowdstrike is, why it is so massively used, why it seems to be so heavily associated with Microsoft and what the hell happened?
Gonna try my best here:
Crowdstrike is an anti-virus program that everyone in the corporate world uses for their windows machines. They released a update that made the program fail badly enough that windows crashes. When it crashes like this, it tries to restart in case it fixes the issue, but here it doesn't, and computers get stuck in a loop of restarting.
Because anti-virus programs are there to prevent bad things from happening, you can't just automatically disable the program when it crashes. This means a lot of computers cannot start properly, which means you also cannot tell the computers to fix the problem remotely like you usually would.
The end result is a bunch of low level techs are spending their weekends manually going to each computer individually, and swapping out the bad update file so the computer can boot. It's a massive failure on crowdstrikes part, and a good reason you shouldn't outsource all your IT like people have been doing.
It's also a strong indicator that companies are not doing enough to protect their own infrastructure. Production servers shouldn't have third party software that auto-updates without going through a test environment. It's one thing to push emergency updates if there is a timely concern or vulnerability, but routine maintenance should go through testing before being promoted to prod.
It's because this got pushed as a virus definition update and not a client update bypassing even customer staging rules that should prevent issues like this. Makes it a little more understandable because you'd want to be protected against current threats. But, yeah should still hit testing first if possible.
If a company disguises a software update as a virus definition update, that be a huge scandal and no serious company should ever work with them again…are you sure that’s what happened?
It wasn't a virus definitions update. It was a driver update. The driver is used to identify and block threats incoming from wifi and wired internet.
The "Outage" section of the Wikipedia article goes into more detail: https://en.wikipedia.org/wiki/2024_CrowdStrike_incident#Outage
Ah, was a bit off. The update disregarded update controls per reddit and I must have misunderstood what exactly the channel update did. I know for the sensors you can set how closely you want to track current releases but I guess the driver update is not considered under those rules. I use CrowdStrike in my day to day but not from the administrative side, sorry for the misinformation. Thanks for the details Gestrid.
100% agree. I haven't been on the backend of managing crowdstrike so I don't know if this is a option, but running a wsuz server and manually weeding out bad updates was such an improvement over rawdogging windows updates.
Yeah but testing costs money and CEO needs new private island, his old one is too small.
And the kids on the island are too old now
Really there's a sub-joke here about how, because no one ever bothers scanning their Mac for viruses since they think they're virus-proof, all the Macs are functioning as the virus farms they've been for quite some time.
Crowdstrike is a cybersecurity company that makes security software for Windows. It apparently operates at the kernel-level, so it's running in the critical path of the OS. So if their software crashes, it takes Windows down with it.
This is very popular software. Many large entities including fortune 500 companies, transport authorities, hospitals etc. use this software.
They pushed a bad update which caused their software to crash, which took Windows down with it on an extremely large number of machines worldwide.
Hilariously bad.
Honestly it is kind of hilarious, with how many people I have had make fun of me for using Linux, and now here I am laughing from my Linux desktop lol
Sure, this time it only affected Windows computers, but Crowdstrike has also broken Linux installs this year:
https://stackdiary.com/crowdstrike-took-down-debian-and-rocky-linux-a-few-months-ago-and-no-one-noticed/
Sure but I'm not using their software. I'm careful about what I allow to have kernel access on my machines.
most Linux "Users" would'nt be that stupid. it nearly always takes a noob in a CEO position (okay, just saying CEO is sufficient here) to blindly trust malware distributers and their ads.
So, do all windows machines use this, or do you have to add this software?
It's separate software; CrowdStrike is independent from Microsoft and it isn't a default component of Windows.
It’s interesting that Microsoft is getting a lot of flack from this.
Yeah, this isn't really the fault of windows.
Windows normalized running third party software as kernel level code.
Third parties love their trojans just being treated as normal way of life.
"Anti-cheats" instead of not being imbeciles while designing protocols for multiplayer, "anti-viruses" which need to run kernel-level and download databases with executable code, video drivers which just can't be packaged with Windows.
One thing I've realized is that large parts of social structure are dependent on cheating. We all want to cheat, so we all agree to a system where cheating is possible, but pretend it's not happening until someone gets caught and then just behave as if nothing happened.
One necessary part of someone's upbringing is honesty. There's an amazingly deep moment in LOTR where Eomer says that Rohirrim don't lie, so they are not easily deceived.
This is not a poetic device. This is how it works. Ponzi schemes usually target people who think they are smarter and more cunning and will gain something from them. And rigged security systems work because most of participants think they are the ones who may at some point abuse those systems, but most of them are the ones becoming eventually victims of such abuse.
I think it's much simpler: people don't know what they're doing, while CEOs want to make more money so don't do appropriate (expensive) practices.
I know it's not simpler because I've tested it in society a few times.
Also if you'd familiarized yourself with, as I said, the ways large-scale scams work, you'd notice this pattern too.
And propaganda.
And it's a common pattern in movies that the "good guys" can "hack" something or do something the shady way, and normies really do think that they'd be more comfortable with having that possibility. They see good secure systems as some kind of digital police state and don't understand that the existing world is much closer to that.
I'm not impartial, of course, my interest in these parts of human psychology comes from studying Nazi Germany, Armenian Genocide, trying to understand why Russian society is as it is and how to fix it, same for Armenian society, and, ahem, engaging in discussions about corruption with people benefiting from it.
In the latter case I was intentionally disallowing all aggressive emotions from my side and such and pretending to be naive and that we are all interested in a better world, and explaining how one can create systems where corrupt people don't multiply like cockroaches, and also arguing from the position of us all willing to solve problems allowing corruption and bendable rules to exist, and noting how stupid it is that someone absolutely unskilled in anything useful can benefit solely from occupying a right place, and that such critical points should be removed. Made them utterly furious and some other people, whom I considered kinda honest, rather unsympathetic to me.
If there is any software you want running at kernel though it is your AV. Not saying Spotify has a reason for running at kernel though.... But running AV at kernel in theory is a better way to protect the machine and you.
It seems to be an enterprise product, meaning normal users might not have been affected. I wouldn't personnaly be able to confirm since I usually have 1-2 month uptime on my windows machine.
Their computers may not be affected, but their everyday lives might be. Some of the affected services include 911, stoplights, banks, hospitals, and a whole other smorgasbord of stuff.
It’s a general security solution. They run on Mac and Linux as well. It just happened that crowdstrike only released the broken update for windows.
They make security software for every OS. My company has it running on our Macs, and Linux servers as well. It just happened to only break windows because that’s what they released the update for.
I was puzzled since my work continued on as usual. I guess my company doesn't use it.
if that's a "good" argument for you, then i've already heared that, and it nearly never really fits. here is another one for you that is an argument as generic as yours: "maybe try eating poo, trillions of flies cannot be wrong, poo is VERY popular food, much more popular than any human food !!! (as in mass per day as well as in its number of consumers)"
I wasn't making a case for adopting this software. Just pointing out that it is widely used, which is why it had such a wide effect.
I think you'll find most corporations would jump off a bridge if they saw their competitors jump.
so i misunderstood. sry then.
and yes, every company running an alltime-ever-in-news-due-to-critical-exploitable-bugs-in-the-mailclient already IS in freefall after that said jump.
Try this: https://arstechnica.com/information-technology/2024/07/major-outages-at-crowdstrike-microsoft-leave-the-world-with-bsods-and-confusion/
it's a glorified anti virus and does a few other things on top.