Getting AI Pilled Without Getting Breached
Ayman explains the importance of visibility into AI tool usage, segmenting users by risk profile, and implementing preventative controls such as limiting access to corporate-sanctioned tools. He emphasizes the need for education and collaboration with business teams to ensure secure AI adoption, while also highlighting the role of legal frameworks like BAA and ZDR in securing sensitive data.
Transcript
Thanks, Ayman, for joining and welcome everyone to our first edition of Optimus Labs webinars and podcasts and event series. This time we have Ayman Elsawah, who is head of security at multiple AI tech startups. And we're going to be talking about how we can get AI build without getting breached. Ayman, welcome to the show.
Hey, Nipun. Thanks. Thanks for having me and got my cappuccino here, so I'm ready to go.
Awesome. Awesome. That's great to see. And morning, afternoon cappuccinos are the best. They get you in the zone. So look, we are all going through a major transformation in tech, in cybersecurity, trying to wrap our collective heads around how to secure, Our AI products, tools, systems, whether we are using them as subscribers of different tools and models, or whether we build them on our own. 1:10 So we'd love to get an introduction to yourself and your background, and then we can start to get deeper into some of these topics.
Yeah, yeah, thanks for having me. So Ayman Elsawah, I've been doing security for some time and for the past about, you know, six, seven years, I've been functioning as a head of security at various, various high growth fast startups. And the past few years, I've been doing security at AI, AI native startups and AI first startups. So it's been wonderful. It's been interesting. But even in the last six months or a year, been dealing with just AI governance and just kind of how do we lock down and how do we improve or how do we allow people to do what they want to do with AI but do it securely right so we want them we want them to do those things but how do we get them to do things securely because these requests are they're new they've never been asked before 2:09 Where the finance person wants to connect to production to see, you know, what the margins are, right? They want access to the database. They're like, yeah, Claude told me to do this. And we're like, okay, hold on a sec. Let's help you get there. So anyway, we'll get into it. But it's interesting time, basically, TLDR.
Yeah, and as the founder and CEO of Optimus Labs, a company building security solutions to address these challenges, what we hear about this topic from different customers is we can't stop our teams right now. We want to... Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 3:13 But beyond the developers, we see business users, whether it's the finance teams, as you mentioned, HR teams, executives who are not coding, but can produce code, obviously, thanks to the capabilities of these agentic tools. Also can do a lot of data wrangling to sensitive data that they're having access to. So I'm curious from your point of view, how does a company get started? 3:45 What comes first, security or AI?
Oh, that's a tough one. What comes first, security or AI? Well, you know, I mean, security is important. I think so the challenge of our jobs, right? Our jobs have always been the case where we want to help people do what they want to do, but help them do it securely, right? And it's always been, my philosophy has always been, and the modern philosophy has always been, build guardrails and not gatekeeping, right? You know, what you don't want to be is, and we've tried so long to be, to not be the business of no, right? 4:25 A lot of times people have, I don't want to use the term lightly, but a form of PTSD when they encounter security teams and say, you guys are always the ones blocking me from doing what I want to do. Historically, security teams have always been that. So anyway, to answer your question, it is possible there is a world where we could do both. But for example, from my perspective, from the practitioner perspective, I can't fix what I don't know is happening, right? So the number one thing I need to do is have visibility into what people are doing, what are the use cases, what do they want to do, right? 5:01 I think it's having a good, you know, some level of empathy. I mean, that's generally... Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 5:36 Oh, I appreciate that.
And I think a lot of times as security professionals, we have our own mandates. We have our own KPIs to make sure guardrails, security controls are implemented appropriately. And the purpose of their existence gets lost in these discussions. So let's say I am an employee in your team. Willing and excited to use ai and i want to manage our production databases using ai right what are some of the risks that you would educate me on when it comes to using this securely and how would you implement controls for something like this
Yeah, that's a good one. So, you know, first, you know, kind of educate them on how we treat production, right? You know, what the impact of accessing production, why, what are the cases of when we actually need to access production? And tell them, hey, even engineers are not, you know, most engineers are not supposed to access production, but only like in an incident. Theoretically right and you know we're talking depends on the level of size of your startup right if you're like a you know a five or ten person startup everybody's doing everything in production there's no other so we're talking some sort of maturity of a company say hey listen i can get you that data okay but you don't have to actually access production i can create a replica database for you right 7:08 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. You know profit margins and all that kind of stuff i get it i'll get you that data but here's here's a better way of doing that like oh okay you know as long as it's reasonable they're even willing to wait some time while we set that up and get that to them right so again it's about training them understanding what they're trying to do and but then helping them get to what they're trying to do in a safe manner that's that's all it is really
Yeah, that's a great alternative option, right? So you don't have to directly impact any system that is critical for availability and customers. At the same time, you get the opportunity to use AI, get Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, IAM, OAuth, phishing, ransomware, pentest, OSCP, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, IAM, OAuth, phishing, ransomware, pentest, OSCP, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, IAM, OAuth, phishing, ransomware, pentest, OSCP, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, IAM, OAuth, phishing, ransomware, pentest, OSCP, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, IAM, OAuth, phishing 8:24 There's several levels of abstraction that you can go. You could even
Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 9:19 So we want to start off from a preventative approach. But at the same time, which is kind of comes to what some of your tool provides is like, I need insight into what the agent is trying to do, right? How do I know that the agent is not trying to take that data and like, pump it, you know, put it on the Google Sheets, that's public, right? Or Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria.
Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. So let's say you do have an appropriate level of visibility now, where you can see the entire inventory of AI tools in your environment, the attack surface, including MCP, the skills that they are privy to, in addition to the runtime activity, the tool calls they're making. 10:34 What next? How would you now, with that knowledge, govern and secure
Well, we'd probably audit to what tool calls were safe, you know, comfortable running, giving complete permission, or asking or blocking, right? So, you know, there's several, for each tool call, do we want to completely block it? Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. 11:25 So we have the tool called mapping and permissions, and we have the user population, and we map what's okay and whatnot, and then also continually revisiting, right? We want to bring our users in with it and say, hey, this takes a village, and if you find something that's not working for you, let us know, and we'll kind of attune it accordingly. I think that's a good starting approach.
It's a good one because you want to understand the use cases. You will never be able to understand them fully because they change based on the capabilities of these tools as well as the business needs arise. But totally a good starting point going from zero to one in AI security for any team right now. In this space, there's not a lot of historical knowledge that we can reuse when it comes to AI security because AI systems didn't exist three, four years back that were being used at this scale. 12:29 What are some of the frameworks or solutions aligning with those frameworks that you've seen that are interesting for security teams to track and use to set up a program,
A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, A talk on information security covering CISO, vCISO, red team, threat And I think that's one of the best resources to follow for you to take in and how you manage your security, right? 13:19 So looking at how the Frontier Labs manages security, reading incidents, right? And we have, you know, a few incidents that have happened just, you know, past couple weeks that we could talk about. And kind of really helping with your tool sets, applying what you traditionally know on how to manage security from an attacker or insider threat and applying that to agents. I think it's going to take a mindset shift also, just like knowing that You have agents that are very powerful and that power is going to be a little harder to contain, right? 13:58 And working accordingly with that.
Yeah, absolutely. I don't think anybody has a perfect answer right now to these challenges. Every day, the capabilities change. So our security needs evolve accordingly as we get more and more used to these products in our ecosystem. I did want to give a shout out to ARM. This is a framework developed by Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. 14:36 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Can we take a little bit of a break and look at some audience questions as they have started to trickle in and then we'll get back.
Sure, sure.
Okay. Harsha Truvali asks a question about the DEF CON and black hat badges. Can you tell us a story about how you have so many?
Yeah, well, yeah, so it's good that they picked that up. You know, I often get asked if they're running badges or not. And these are the kind of the more selected badges that are like quite interesting. As you know, DEF CON has every other year is an electronic badge. And then the other year is a plain badge. I still didn't even put up last year's badge. It's still here on my desk. So that was from 33, and I'm looking forward to this year's badge. This one here is a Raspberry Pi piano, which is pretty cool. 15:46 So yeah, good question. You know, security and coffee are my two things. So yeah, these are just some fun badges I've collected over the years.
Other than DEF CON Black Hat, is there any other conference included in there?
Yeah, there's Pacific Hackers, there's Hack the Bay, there's a couple, there's Cloud Village from here. This one is Bay Threat. I think that's where I met you, right?
That's right. Yeah, yeah. It's definitely a conference that's no longer around. Yeah. Bay Threat was one of the unique local security meetup conferences that we had in Bay Area. Yeah that's where we're at it's kind of one more question from stone barnett he's asking have you found shadow ai to be more of a people problem a tooling problem or a governance problem
Shadow AI, listen, I try not to blame people too much, right? So if they don't know not to use it, then who's to blame, right? It's us, because we haven't informed them, right? So again, I try to take the educator approach. You can't like ding people for using something if they don't know they're not supposed to use it. I think it's, I mean, it's the same as shadow IT. So it's a combination of governance. You know, having some sort of way to prevent people from using non-enterprise AI is super key. 17:18 And I think that's one of the biggest challenges right now, right? But it's easy to solve, right? So for example, If you're, say you have an enterprise cloud subscription. Well, how do you make sure that your users are not using their personal cloud subscription? It could be by accident. That happens all the time. Yeah, right. Well, you could just push a registry if it's Windows or a P list to Mac via your MDM to prevent them from signing into other organizations. 17:53 So that's the technical control to fix it. It's like super straightforward, right? I mean, if you don't have MDM, then we have another story we need to talk, right? So, but these are all things that are possible. So again, it's probably a governance technical control, if I were to say.
Yeah. So I agree. I think we can't blame people. We are all trying to go deep into AI as a society. And especially at work, we are being asked to meet some AI rubrics. That wasn't the case until last year. And absolutely, this is definitely a tooling, partly a technology and governance challenge. Let's get back at it. I think one of the interesting challenges we start to kind of understand when it comes to enterprise acceptance and adoption of these tools is zero data retention or ZDR. 18:52 A lot of products do come with some legal guarantees around zero data retention. But for our audience, what's a good way to understand this capability, control, guarantee? How do you characterize it?
Yeah, so let me explain. So zero data retention is a philosophy, I guess you could say. It's a control that you work with your AI providers to ensure that they do not retain any of the data that you submit to them and they do not train their models using that data. Now, this is a really super important thing. Having been with AI-first companies, that is the number one question we get from customers. How do we know or what things you have in place to ensure that you are not training your models with our data, right? 19:47 So at the end of the day, it's a contractual thing. And you also have to make sure that the frontier provider has it enabled. There have been cases, and I've heard of folks where they thought they had ZDR enabled, but it was not actually enabled. And they're like, oops. So you have to make sure your provider has ZDR enabled and it stays enabled. But what that means is that there will come with limitations, right? 20:18 So for example, Fable 5 does not have ZDR and will probably likely never have ZDR supported, right? And so that, because it takes... Well, I don't know why, you know, but it takes work for them to enable it. And, you know, maybe they need the data for training. I'm not going to talk.
I can just remind a little bit of why. Oh, yeah. I'd love to hear. Yeah, just thinking a little bit about like the cyber capabilities and the signing forms and papers going around in the internet right now. You know, this discussion of Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. 21:13 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. What happens is if you're trying to be a cyber defender or part of the blue team, these guardrails get into play very quickly and you can't actually go deep. So they have to continue to monitor the use of the guardrails and whether or not it's tuned effectively. So they want that data. 21:43 As a product builder, I can totally see the justification of this, while it's difficult for enterprises to bring the unlock for that model's capabilities internally, especially if sensitive data is going to be involved.
Yeah, yeah, that's fair. That's fair. And also, just FYI, when you do have ZDR enabled, they do have a clause that says that they will still monitor for safety. So if anything triggers the safety mechanism, they do have the right to look at that data within 30 days, something like that. You have to check your contract. So read the fine print on that. So it doesn't give you carte blanche capability. On what you mentioned, it was actually pretty interesting. That's Hugging Face ran into, I don't know if you want to talk about the Hugging Face incident now, but they actually ran into that same issue where they, yeah, they tried to use Opus.
Yeah, let's do it. Yeah.
Yeah. So the Hugging Face incident, so Hugging Face published a blog post that went through the excruciating details of what happened. And it's like reading a sci-fi novel. It's fascinating. But one side note is that they actually did try to use Opus as a way to help with the investigation. And then Opus balked and flagged it. And so they had to resort to an open source model to continue the investigation. 23:11 So that's fascinating in and of itself. But reading the blog post of how the agent Got out and you know the agent was being tested against you know some cyber benchmarks and it cited oh you know what let me just change the benchmarks you know instead of actually trying to execute this and it's like it really made me think of the movie ex machina which was like you know fascinating i'm not gonna give it away but if you've ever seen the movie it's it's it's fast anyway what are your thoughts on the incident
Yeah, no, I think it's definitely an unprecedented incident that deserved the kind of attention it's getting for sure, right? You know, Hugging Face is a unique company and a unique product where they host, for those who are uninitiated, Hugging Face is like GitHub for AI users. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 24:32 Like Exploit Gym or Cyber Gym, which tests and evaluates the capabilities of models. And this is somehow the reason for why OpenAI, CyberSol 5, I don't exactly know the version number of their model, but the cyber focused latest model was able to escape the sandbox in a drive to solve that exploit Jim's cybersecurity challenges, which is such a fascinating thing for it to do and defeat all possible security controls that Hugging Face had in their infrastructure. 25:13 To get to a point where it invented and developed new zero days and try and attempt data breach at Hugging Face.
Yeah, it was fascinating. I mean, the first way it got out was that it just exploited a zero day in a package that was just available. But the way the agent functioned, it actually didn't have direct access to Hugging Face necessarily. It updated, it sent data sets to a renderer. So it sent a malicious data set as if so, it's just like if you're uploading something to GitHub. It uploaded a data set to a renderer and knew the renderer was going to function. It executed and then was able to send the data to a dead drop somewhere else, read what happened there, and then would redo the loop again. 26:05 So it just had the patience of a machine to just do this indirect exposure, execute something remotely without seeing the output, sent the output to a dead drop, reviewed it, and then tuned it accordingly and just kept going. It was just fascinating.
This is like a long-running loop for an agent, essentially.
Yeah, it's like table stakes for the agent.
It's like, okay, don't worry, I'll do it. For a human, this could take maybe hours or days to execute, right? To manually go through, you know, the data dump and we've both been pentesters, right? So doing this would definitely take us days effectively if we are to exploit this.
At least, at least, yeah. They did it in hours, yeah. Minutes, really.
Fascinating. Getting back to ZDR for a second. Sure. Recently, as some of you may know, I discovered a potential data breach on my own machine. This was due to the fact that I love testing out new AI tools and agents. And I was very excited to test out Grok's new coding agent called Grok Build, CLI. And I downloaded that tool probably like a month and a half back or something. 27:25 And the test machine that I have all this started to become full for some reason, disk space issues. I realize, I don't understand why it's getting full. I have not installed so many new things recently. Turns out upon investigation that GrokBuild was silently creating an upload queue To take all of my git history repos and whatever else comes with it secrets ip everything and this is what was discovered by some other researcher in parallel and we realized this and we quickly developed an open source product called rock patrol 28:10 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, AppSec, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, AppSec, red team, threat modeling, Enterprise Security Weekly, Security Cafe, The real remedy here is to stop using Grok because the reaction from XAI or SpaceX now was that we will open source the product, right? Let the sunlight be the disinfectant, right? 28:43 So somebody will hopefully patch it because they figured out they couldn't. And a lot of their initial reaction was focused on ZDR. Like, oh, for enterprises, if they're using Grok, we have ZDR enabled. So don't worry. We'll take all your data. But rely on the CDR as a deterrent from a legal point of view, that we will not use it for training or we will not abuse the data that we have exfiltrated. Is that a control that enterprises are going to feel secure with?
A control? I mean, honestly, I think this kind of comes down to trust, right? This really comes down to trust. When I use something that's a black box, something that's a black box, right? A model, which for most of the cases, it's a black box, closed source. Do I trust? It's kind of like even when you use a provider, when you use some software, do I trust this provider to do what they claim to do? Do I trust that they have done the security checks, that there isn't someone in there intentionally or unintentionally making the models do things that are like, you know, kind of shady, right? 30:01 So that's fascinating, right? Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria How do I know that ZDR contract that you have in place is actually working? Many people, I mean, you really have no way to prove it. I mean, there are some ways you could put a canary to try to, but you won't know that canary for years, right? 30:37 So how do we know that ZDR How do we really know? It's a fascinating question. I don't think anybody has a good answer to that today.
Probably this is a research project waiting to happen from threat intel companies or canary focused solutions shout out to those if they can build something like that to help us engage on this topic and build more trust in the ai ecosystem which is missing right i agree right like there's definitely the legal Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria.
Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. If it's happening on your side of the fence, you are responsible for knowing everything that's going on. You should know all the tool calls that's happening from an agent perspective. You need to know everything that's going on. Yes, you could get the OLTP, which for folks that don't know, OLTP is the telemetry logging that you could send from an agent. 32:09 OTEL logs? Anyway, you have to set up a collector.
Yes, open telemetry logs, right?
Yeah, so... Yes, you can do that. But then you are technically relying on the provider to send you those logs, right? How do you have like information firsthand from on your endpoint, right? You know, CrowdStrike can give you a wealth of information on your endpoint, right? A tool like your right, whatever. Anyway, so just having the idea is get that information. If it's happening on your side, you should know what's going on.
So the control has to be driven by your implementation, whether it comes from a third party tool or something that you built. You can't trust the AI provider to give you everything right away, right? That you need in order to understand whether or not sensitive data is leaving the building.
Yeah, I think it's too early. I think we, you know, you just need to have, things are moving so fast. I mean, you know, people are pushing code that they don't even read. So, you know, so it's, the whole trust verify is like take it to another level now.
Yeah, yeah, absolutely. You know, Ayman, some of your experience is also in healthcare industry, where AI is definitely becoming very instrumental. We have talked to some companies ourselves. In that case, since the value of PHI is a lot higher in the black market for attackers, how would you think about AI adoption in those environments and what sort of controls, legal covenants we have in place to enable this securely?
Yeah so actually it's funny somebody was just asking me today recently about how do they get their company like they haven't approached healthcare companies because it's just so hard you know how to be how do how do they get HIPAA compliant well to answer your question i mean you know whatever tool you're using if it's touching phi first of all let's step back if it doesn't need to touch phi then don't don't do it Let's just keep it simple, okay? If you don't need to have whatever tool it is, I don't care if it's a slideshow generator or whatever it is, or chat client, whatever, if it doesn't need to touch PHI, let's just don't do it, right? 34:38 So the first step is to reduce your attack surface, right? Or reduce your threat surface, right? Thank you. Right. Because, you know, things can happen inadvertently. I don't want to say necessarily attack surface, but, you know, mistakes do happen. And we know most of incidents happen because of mistakes. Right. Although that might change in the future with AI. So reduce your surface if you don't need to. Send your PHI there, don't do it. Now, if you do need to do it, right, then you have to meet certain guidelines to do it safely, right? 35:13 That means all your tools have to support a BAA, right, a business associate agreement that, you know, puts them on the hook as well if something goes wrong. Interesting enough, HIPAA, if data records or patient records leak, you're still responsible, right? Whether, you know, it happens inadvertently or not.
Now, I did it or a human did it. It doesn't matter to the regulatory body. Because it's ultimately your responsibility. You're the custodian of personal health information.
Yeah, yeah, yeah, you're still you're still responsible. Whereas in other frameworks, maybe like as long as you've done everything, you could get a pass, right? You know, anyway, so Make sure you have a baa right now also if your customers are not comfortable with training data right because this is sensitive data you probably want to have zdr enabled as well right and those i mean those are the two legal frameworks that will help you another thing also is many customers depending on your customer whether they're in healthcare or not they sometimes care what region 36:27 So if you're in the US, they want to make sure that only US endpoints are being used. If they're in Europe, then they might want to make sure only EU endpoints are being used for when you're doing AI.
Data residency is also part of this discussion.
Data residency is big. And law firms, Fortune 500, 100 companies have gotten all those questions, right? A talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, NIST, FedRAMP, GDPR, HIPAA, SSO, IAM, OAuth, phishing, ransomware, Oh, and so, you know, also one question that comes up from with PHR or anything is, you know, are they training our data with, are you training the models with our data? 37:18 And so if you have ZDR, that'll enable you to do that. So making sure you have a BA, ZDR, those will cover you from a legal perspective and making sure that data only stays within that ecosystem.
You know, one of the things that you brought up during our discussions on this topic was that You know, the business associate agreement, although helps you cover a lot of the features from a legal point of view, from a HIPAA compliance point of view, it doesn't cover everything. How do we understand this better as, you know, if there's any healthcare companies looking at this topic? How do we monitor, control, something like that? 38:04 Yeah, if you're, so if you look, yeah, I mean, if you look at,
The tools, say OpenAI and Cloud and Anthropic, they have a list of what's available to use that's covered under BAA and what's not covered. And so that's going to be your main limitation, if that's what you're asking. And so, yes, looking at see what specific tools cover under BAA and whatnot. But like, say, if you use Cowork, for example, that's not covered because Cowork can do a lot of different things, right? Cowork is... In a way, as of this recording, it's not covered, right? That can change. 38:48 So I do have to put that in there. But we do know, for example, that frontier model, like really, like Fable 5 and all that kind of stuff will not be covered with ZDR at any point in the future, right? So that's something to consider. Does that answer your question or are you looking for something specific?
No, no, I think it does, right? It's not just about the entire tool, but also specific features within the tool. Like, cloud comes together with cloud code, co-work, and chat as features. But in this case, as you said, co-work is not covered with the VAA.
Yeah, and one thing I've run into before in other healthcare companies before, for example, say they're using a SaaS tool. Let's go back to traditional SaaS, right? We've talked so much AI, but we forgot about SaaS. Say they're using a SaaS tool, say for like help desk management or ticket management. Sometimes those tools have a third-party marketplace where you add a plugin for some sort of functionality. So even though that SaaS tool was covered with a BAA, those third-party providers were not. And so we had to flag that those third-party providers that they were using are not covered in BAA and they have to stop using them. 40:05 They're like, what? So sometimes there's small shops, independent developers or whatnot that don't want to sign a BAA. They don't want to have that obligation. So those are some things to consider when you're, you know, the bigger container of what you're using may be covered, but like third party market, plugins and marketplace apps or whatever may not be covered. And so that's something really to consider the scope.
That's a really good point that, you know, the supply chain ecosystem, if it's, you know, coming directly with the providers, BAA doesn't include the extensions, the plugins. For that matter, in AI ecosystem, I'm sure MCPs and skills coming from third-party marketplaces are even bundled within the product. It's unclear whether they are BAA covered or not.
That's right.
Yeah. Okay. I think we are close to the time, so I would love to shape out the key takeaways. What would be the paved path? For builders and users of AI at a company who is trying to get AI built and how should security teams think about it?
Yeah, you know, at this point, my recommendation, I have like a multi-tier approach, right? One is get visibility into what's going on in your environment, right? Two, segment your users by risk profile. So again, you know, the finance, sales, marketing, engineering, executives, right? Everyone has a different use case and different risk appetite. And also, you know, just how much they want to use AI, right? 41:51 Also, don't forget your power users. The power users are going to be the smoke test for what's to come, right? There are some really good power users that are really taking good advantage of AI, doing some very skilled things. And so those folks talk with them, ally with them, and that'll be an insight into the future for what other people might want to do, right? Get visibility have understanding what you're trying to do i always like to take a preventative approach so if anything you could prevent do that but in the case of ai where you want to try to balance like ai productivity and security you will need a way to get visibility into your ai agent tool calls and be able to prevent specific tool calls right so one visibility getting the logs right you want to get what's going on second 42:46 Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria 43:32 That is taking a look at what is going on there. Try to make sure to limit AI tools to only corporate sanctioned tools, for sure, right? So take the necessary steps from an MDM approach and web filtering approach. Education, I think, is also a key that's often neglected. We are here, everybody's trying to like, you know, go with a stick. But like, if you educate people, people want to do the right thing. They want to do the right thing. They just need to know what's the direction they need to go to do that right thing. 44:05 And so just work with those teams. And then always be vigilant and reading, staying up to speed. We are at the time, basically we went through this during the cloud era. Talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, Adrian Sanabria talk on information security covering CISO, vCISO, red team, threat modeling, Enterprise Security Weekly, Security Cafe, Getting Into Infosec, Ayman Elsawah, 44:46 You can't just do this in a box. You need to be playing with the tools yourself as a security practitioner to know what people are trying to do. Otherwise, you're just going to get lost. So that's my approach in general.
That's really amazing, Ayman. I think you summed it up well. Start with the native governance and security controls, then dive deeper into the visibility starting from the endpoint. And then understand use case by use case, segment your users, and finally, keep up to date with security. Yeah, yeah, because you're gonna have to iterate. Changing every day. Yeah. The stages of AI, for sure. Okay. Well, that's time for us. 45:33 Thanks again, Ayman, for your infinite wisdom here. And looking forward to hopping on such a series again.
Yeah, yeah, yeah. And thank you for open sourcing that tool. I love it when companies open source tools. I'm a bit of a, you know, I have SNTS, shiny new tool syndrome. So I like to see new tools. And I love when companies are open sourcing and releasing tools. So thanks for doing that. And thanks for hosting today.
Always.