What You Need To Know About Restoring From A Backup

Hey, everyone. I am an also CEO of Cloud Security Labs. We do vCISO security advisory for high growth startups and happy to be here based out of California in the Bay Area. And we help companies with their security programs. And my background before that was also in enterprise with a specialized focus in cloud security, IAM and security operations.

The way I like to tell folks, or the way I like to frame it is, if you lost data tomorrow, today, what would you cry over? What would you, you know, lose sleep over? Just start from there. You know, what are the type of data that if it was missing tomorrow, you would, you know, lose sleep, get fired over, I'm not sure, right, get sued. So start from there. It's pretty, pretty straightforward question. You know, there's some data that you can restore back and repopulate, but there's other data that's, you know, specialized and analytics and metadata that you've taken from your customers, around your customers. 3:22 So without getting to specifics, that's really, You know, just identify what you would lose sleep over.

So if you're talking about system operations and critical infrastructure, I am very biased towards cloud and automation. So, for example, if you have, say your AWS configuration, if you had set it up all manually in the AWS console, well, it'd be easier to get it in Terraform, for example. So, you know, diving into the weeds here, but getting a Terraform, it's all in source code, hopefully using, you know, say a source code platform that's online, a SaaS platform that's backed up like GitHub, and that's all tracked and, you know, you have your configurations there. 6:31 Active Directory, you know, hopefully you have multiple ADs or multiple, you know, domain controllers leveraging Azure AD to kind of, you know, sync, things like that. So, you know, I'm a big proponent of cloud. I've kind of, you know, I've had, I'm drinking the Kool-Aid. So, you know, that's going to be my approach.

Yeah so along those lines I mean I think it's good to elucidate the different types of data backups and different levels that we can have and you know everybody here is familiar with this but other folks might not be so for example having high availability so you know taking that system and making it fully redundant right so you have that system and then there's another backup system and another tertiary backup system where you know it's so you can have either high availability so 10:23 Active for example or active well it's the new word but you know so basically dormant so the second site would be dormant and so that you can restore and you know again like if you know say remember you might be good to whiteboard right take a look at your configuration and say hey if this machine completely disappeared can we do what we need to do right so high availability 10:54 A disaster recovery right so if a disaster happens if this if it's an on-prem you know and it's stored at HQ if HQ gets hit by a storm I mean we all think we're all thinking ransomware but also weather and storms and floods happen right so if this equipment is destroyed what's gonna happen you know do we have a full complete backup of somewhere else so

Yeah, so I'm going to take it from the cloud perspective. There's a story of Codespaces that I like to teach my students about. A long time ago, this company, they provided secure code backup for their Git repos. And at the end of the day, an AWS key with admin or root privileges got compromised. And completely deleted everything in there so the lesson from there is you know just so to go back to the question you know if you're if you're in the cloud using you know configuration backup using infrastructure as code okay the cloud provider is taking care of some of the hardware items but they're not they're not protecting your software so if you have everything in one region 14:11 A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. 14:51 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria.

And then something to add to that, I'm sure as part of your plan is, you know, that data should be at least 50 miles apart, maybe even more 75 miles apart.

So we have all of those in place. Oh, go ahead, Ayman. Go ahead. And then if those backups, we haven't talked about encrypted backups, but if those backups are encrypted, don't lose the key.

Yeah, so. When you go to test your backup, you'll know whether you can restore it or not when you go to decrypt it. Don't lose the key.

So one of the first things is recovery time objectives. RTO, fancy word, but basically what it means is, and it's going to take collaboration with the business because you might have an idea as say the VP of engineering or the CTO or CIO, And the CEO might have a different idea. So how long is it going to take to restore that data back to an operational part? So your business is back in operation. So, you know, is it going to take three hours, five minutes, a day? 24:49 You know, have you considered, you know, thawing it out and removing it from cold storage? And if say you have it in Amazon Glacier, right? How long is it going to take to restore it? Check out their, you know, SLAs. So you really need to decide how long that's going to take. Once you have that, then test it. Find out, okay, I'm going to restore it. And you're actually going through the procedures. Oh, what was that command again? How do I restore this? Or how do I have that all documented, right? 25:19 So make it easy for the engineers so they're not fumbling and you're in crisis mode and not even sure what the command is to restore properly. And then you realize, That when you restored it, you know, you didn't restore a full, you didn't restore it properly because you were missing some attribute in the command or something like that. So I'll start off from there and see if Kathy has anything else.

Yeah, it's an interesting question. If you have sensitive data on there, yes, I would say so. If you're backing up social security numbers, date of births, any sort of restricted information, especially to tape where you're giving it to a third party and shipping it somewhere, because tapes get lost, media gets lost all the time. So absolutely 100% encrypt. When you're backing up to S3, technically it's encrypted on the back end. 32:53 So it depends. Honestly, it depends on the type of data. But in this day and age, 2022, encryption is kind of ubiquitous. It's kind of expected of you as an organization, depending on the type of service that you do. I mean, if it's pictures of kittens, then sure, don't encrypt it. Okay, sure, that's fine. But, you know, hey, you know, that might be some, you know, you might have to back up these images for thumbnails or something, you know. 33:25 Hey.

Yeah, it's kind of expected. It's kind of expected now. I always err on the side of encrypting. I guess, you know, it would have to be a certain exception on what not to encrypt. But Kathy, you're going to say something?

No, sorry. I have a story. Oh, sure, sure. So, you know, there's a company, a blockchain company I worked with, that would generate data. So if it's like, if it's generated data, where it's, you know, data that you can generate again, if it disappeared, you know, you know, then do it, do they need to encrypt it at rest? You know, it's something that, and it's, if the data was lost, it's kind of useless to whoever was, I mean, it really depends. 36:10 You know, if it's like really core to your IP, then encrypt it. In fact, I was getting the question, you know, should I encrypt it in transit? I'm like, okay, that's actually probably more important than encrypting it at rest. So, you know, your mileage may vary, I guess, depending. And I think the newspaper test is great. Like if you lost this data, you know, would your customers be really upset? Would your CEO be upset? On whether the data was lost or not. And we're talking petabytes, petabytes of data, leveraging, you know, if S3 is already encrypting it, or you could use a KMS key and just, you know, encrypt it and there's no performance impact. 36:52 You know, so, you know, I still hear, I still hear these days, encrypting it is going to affect performance. And I have to push back sometimes and say, okay, well, can you show me how, can we do a test? I think there's still a lot of perception within cybersecurity and within information security and information security practices that are still a holdover from the days before, right?

Yeah, there's a few ways. So I mean, without any budget, you can Try to run checks to make sure you have the right data. So taking that image backup that Kathy mentioned, running an integrity check. So do a hash of your backup and do a hash of your live server, maybe the partition image. And then all the way to asking for a budget. So asking for that backup server or that HA server. 40:44 So go to your boss or your management and say, hey, This is how long it's going to take or to restore or whatnot. And we need to make sure, you know, our servers are backed up. Can we buy a low-end HA server or, you know, whatnot that can, you know, function as both high availability server and, you know, way to restore. And show them you know here's the cost of not buying it right here's the amount of hours that we're gonna have to spend to just make sure that this backup works and whatnot but if you really absolutely can't have that yeah you know trying to use the available tools like running integrity checks restoring to the cloud as kathy mentioned doing spot checks you know of files so pick important files every once in a while and see if they're there and they match

Snapshots, snapshots, and you can take snapshots, daily snapshots, and all this stuff. Yeah, it's a really good point. This is great. I love this.

Pay your bills. So I've seen, so there's a lot of like third party, small like cloud backup solutions. I'm not talking like AWS or the bigger ones, although those can be included. But, you know, I won't name them, but say you're using one of those services, which are great. They even do like full bare metal or SOAR. And then your credit card expires and you realize that you haven't paid your bill and the account just disappeared. That happens. So yeah, pay your bills. 46:06 If you're backing up to maybe Synology and then that Synology is also backed up to S3, make sure your S3, your Amazon account is paid. It's so funny that I've seen some of these, oh, they go to restore or I'll ask the IT folks, hey, Are you backing up? Like, oh, yeah, the so-and-so account expired. I'm like, well, why didn't you tell me? What's going on? Like, I'll give you a new credit card. What's going on? So, yeah, pay your bills. The system configuration piece, so important. 46:38 Like, you know, I love bare metal backups. If you are still using bare metal, have a bare metal, like, down to the file level, it's just so much easier to restore. But then, you know, leverage virtual, you know, VDIs for that. And then, you know, store your keys in a good place. Like, 1Password's a great place to do it. You know, create a vault with a few folks and store those keys in a special vault. 47:09 Please don't, like, you know, have it in a printout underneath, you know, in the CEO's drawer or something like that. So, please have your keys somewhere. It's important.

Yeah, I'm on LinkedIn. I'm on LinkedIn. You can find me there, Ayman Elsawah, or if you just do Ayman Security, I'll probably be the first one. And if you have any questions, ayman at cloudsecuritylabs.io is my email.

I'll be over here.