Crowdstrike Global IT Outage Explained! (Hint: Not a Cyber Incident!)

Hello and welcome everyone. My name is Ayman Elsawah and I am here at the Security Cafe. I have my coffee. I take complex security topics and try to distill it down and make it easy for everyone to understand. And today we have a doozy. We are in the middle of a global IT meltdown and we're just recovering from it. A lot of you may have heard of the CrowdStrike outage or the Microsoft outage. I'm here to explain everything about it and dispel some myths from it. 0:30 So let's get into it. As you may or may not know, you're maybe on vacation. Maybe you're somewhere on the beach and just unplugged. Good for you. If you are, that's awesome. But the rest of the world has experienced a giant meltdown. And what is this meltdown exactly? We're talking about 911 calls, centers, airports, and flights being canceled. Banks and atms not working almost everything has been affected all across the board and there's even things we probably don't even know you may have seen even digital signs experiencing a blue screen of death this is all due to a bug in software that got rolled out on friday and i'm here to explain what's going on 1:10 Also, in case you don't know, IT people have been working around the clock to clean up this mess, to try to restore everything back together. And it's such a complicated restoration procedure. I'll also talk about that as well. So my heart goes out to the IT folks right now that are or have been or will be dealing with this in the future for time to come. Let's get into it. All right. So what happened exactly? CrowdStrike, which is a anti-malware software, right? Or anti... Typically, we heard of antivirus, same thing. 1:40 It's software that's designed to sit on a system to prevent cybersecurity issues from happening, to prevent people from hacking machines, right? CrowdStrike is an anti-malware software. What happened was it made a software update. Now, why was the software update so bad? Let's just step back a little bit. This software CrowdStrike runs with elevated privileges. It runs with as an admin. So it has root or system level permissions on devices. Why? Because if it's trying to protect an attacker from doing malicious stuff at that level, it needs to also fight fire over fire and be at that level so that could prevent these kinds of things from happening and prevent attackers in their tracks. 2:24 This software, and it's always updated, so this kind of software always has to be updated to detect the latest and greatest attacks that are happening out there. On Friday morning, an update went out, and the update was a bug. It went out incorrectly. There was an issue with it. And it crashed Windows. It did not affect Linux and Mac OS, but it did affect Windows. And in case you don't know, there are millions and millions of Windows machines out there running critical software that you don't know. 2:57 We're talking about ATM machines, flight scheduling. Windows has been out there for a while, and we'll talk about that in a second. These Windows machines just completely died. They just completely went offline. And to recover them, it's a little bit of work. So let's make clear, this is not a cybersecurity incident, okay? This is a software bug by a cybersecurity software company, okay? So it's a bug that got rolled out. And bugs happen all the time. They usually don't have such a big impact, but they do sometimes have impact. 3:30 It could be a software bug or a process bug or a variety of different bugs out there. Things happen. There's usually a process to identify and squash these bugs before they make an outfit to the world. Okay. It's called QA and. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. 4:19 It's bananas out there. In any case, so why did this affect so many machines? The fact of it, like I said before, is there are Windows computers everywhere. The Windows has been the business software to go to for the longest time. Now, there are Linux machines to do different things out there and maybe some Mac OS too. But it's just a fact of life. Windows is out there. There's some software that only works on Windows. Even legacy software. Sometimes you have legacy software that was only created on Windows. And they're sitting there in a data center somewhere doing some obscure thing. And then if that machine crashes or dies, we're talking about factories as well. 4:53 Windows machines are everywhere. It's just a fact of life. And as machines get viruses. And so unless you disconnect it from the internet altogether and nobody touches it, Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria And as I said before, this software needs to run as root or system so that it can do its job. 5:28 So what's the impact right now? Consider this machine completely offline. So what happened when this update rolled out, it crashed Windows completely, like really bad. It didn't just give like a pop-up error. It just completely took the machine offline. And worse, this machine, if it's encrypted, which it should be as any company If the machine is encrypted, to recover this encryption key, you have to go and pull it from somewhere. 5:59 And a lot of IT folks are scrambling to pull these encryption keys. So in order to recover these machines, you have to go there and change this file, remove this file or change it or whatever it may be manually to this machine. So we're talking about either remotely or whatnot through a console changing this file so that this machine can't start because this is a driver level system update that happened. When Windows goes to call up and reads this file, it crashes and just can't go It's crazy. People are working nonstop. 6:31 I wonder what folks are doing for those who can't even access the BitLocker key that are out there. Hopefully these keys are available, but I'm going to guess that there are systems out there that folks are not able to find the BitLocker keys. And so what does that mean? They have to redo the machine from scratch, complete disaster, and restore it from whatever backup they had. Hopefully they have backups. So did I say that this was a manual process? It is excruciating for an IT person right now to be restoring this. 7:01 And this is affecting millions of machines worldwide. So what's interesting is how are we going to react in the future, right? How are we going to react? And I find this really interesting. Are we going to trust things less now? I think so. I think we're going to trust things a lot less. We're going to question some of the practices that we're doing. I know I remember a long time ago and some banks in high risk environments actually rigorous test So it's standard practice that you don't automatically update systems that are in production. 7:35 Antivirus signature updates are things that we're generally allowed for. So what's going to happen now? Not only are systems, hopefully tech folks all over the world are going to make their systems a little more resilient. And so think about disaster recovery like a disaster can happen from anywhere, right? Whether a data center dark or the system gets deleted by accident or something like this happens that impacts a machine from an unknown source. Hopefully systems will be designed more resilient. Another thing is obviously people are going to demand answers from CrowdStrike. 8:06 So there's probably going to be a public hearing. They've been really good as far as communicating some things out there. There's been apologies and all that kind of stuff. So there's a lot of hate also, of course, which I don't think is healthy right now. That's something to be said. But what's interesting is this could lead to more insecurity because companies might not update or even install anti-malware software on their devices and machines anymore, right? If it's a critical machine, they may take the risk that this machine could get hacked versus worrying about this machine being subject to an issue before. 8:42 And by the way, this actually happened before with Linux a few months ago. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. If in case you haven't noticed, these things have been happening more and more, right? 9:14 Not necessarily software bugs, but the dealership software issue that happened several weeks ago where so many dealerships went offline because of a ransomware attack. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 9:52 Hold our companies a little more accountable. And three, just take into consideration that these are, these events are going to happen more and more. So leads back to point number one. Anyway, I hope I broke this out for you in a simple and easy to explain method. If you learned anything new today, please feel free to share this video and appreciate you. And we'll talk again next time. Thanks. Bye-bye.