Hey, everyone, and welcome to this week's episode of AI Security Ops, where we are going to talk about an incident that occurred around March 24 in which production systems running LiteLLM or a Python library that's downloaded roughly 3,000,000 times per day started crashing. CPUs pegging out, containers killed by out of memory errors, and originally looked like just kind of a bad release of this package, but turned out that it was an entire supply chain attack that was not unique just to LiteLLM and actually started further up the chain. But before we start talking about that, let's talk about Black Hills information security. If you and your organization are in need of any security services related to external, internal pen testing, web app pen testing, Zoom compromises, physical pen testing, social engineering, anything that you might need for your security to help shore up your security posture, check out blackhillsinfosec.com. And we also offer training through our training branch of anti siphon training, where our consultants who are doing this work day in and day out are kind enough to put together their knowledge in a nice neat little package that they, deliver through our training services that are at a very affordable price.

So check out, antisiphontraining.com. Alright. So what is LiteLLM? Why do people care about it?

So LiteLLM is an open source Python library by Verrier AI. It's kinda like a gateway to route requests across LLM providers. So you have some input that's going into an LLM, and it'll route it to OpenAI, Anthropic, Azure, Google through like a single kinda interface. And apparently, it's used in a lot of places, local development, CICD pipelines, production proxies, agentic tools. And as someone who is working on automation and agentic AI tools, I will I will say that this is quite terrifying.

Like, I'm yeah. I I I'm questioning my own, like, methods and and procedures now. But

Yeah. It's scary stuff, and we'll certainly return to that point as we dive a little bit further into this.

Yeah. Because, you know, when you're storing these like, you know, secrets for, you know, using like I'm using Bedrock and the keys that I have, I mean, they they're limited to Bedrock on AWS and they can but they're not limited in how much inference they can provide. I mean, they got stolen, oh, wow. It doesn't take long to rack up a lot of money. Just ask John Strand.

Right? And so because, you know, it sits between, you know, apps and AI providers, it's got a whole bunch of not only, like, inference keys, like, for stuff, but, you know, API keys, cloud creds, like, oh, SSH keys involved. Just yeah. It's like once you get in to the underside of the CICD pipeline, there's a a lot of fun stuff in there.

Yeah. We talked about this quite a bit on the news. So we're gonna be hearing more about this for a while because it it not only tackled the vulnerability not only impacts specific versions of the LiteLLM library, and and it has malicious code in that, but it also has trivia, which was, I forget who it was. That's right. Aqua Security trivy.

So between the two of them, we've got assets that are crossing GitHub actions, Docker Hub, NPM, OpenVSX, and PyPI. All of these things have been impacted. And because it's a supply chain attack, that's one of multiple reasons why this is gonna have such an impact. And we've been saying, this is exactly the type of thing that we've been dreading is the use of of tools against these libraries now and, you know, basically, two separate libraries, and we've got this massive impact.

Yeah. I mean, I feel like, for a while, the conversation was kind of, you know, like, supply chain attacks are more of a theoretical type thing that, sure, yeah, they could happen. As Derek pointed out when we were talking in in kinda pre show banter territory that it was more of like, oh, yeah, nation states. They're working on that level of stuff, but that's not what we're seeing now. What we're seeing is kind of these, the these pop up groups that are fairly recent, not necessarily unknown if they're necessarily tied to nation state activities.

I mean, they could be, but nonetheless, one way or another, we're showcasing that these supply chain attacks are viable methods that these threat actors are using?

So I think two things. One, I think supply chain attacks, these kinds of things have been around for a while. Mhmm. Right? So I think that, you know, looking at how people use Python and Python libraries, like, I mean, who among us has ever done a PIP install as root?

Right? I mean, you're lying if you haven't. Right? And and so, you know, I think this kind of thing has been around for a while. You know, I and when I was in the master's program, like, our one of the research projects we did was essentially, like, looking at all of PyPy and creating a graph network.

It was pretty fascinating. We found malicious packages, like, stuff that's what not what the purpose of it was. We were, like, measuring things. But anyway, like, we found malicious stuff just looking at that. So I think this kind of thing's been around, but I think now where previously it would take like a nation state level of effort to go find this kind of stuff, now, especially as someone who's been working on, like, using AI to look at external attack surface, the barrier to entry is way down now.

And I think that you can create agentic type processes with AI to go find these kinds of things and take advantage of them on a much cheaper and wider scale. And I think that this is going to be a, you know, a fun year or two or three journey before it all shakes out.

Yeah. And because LiteLLM and Trivia, they're libraries. They're they're impacting additional pro software. Like you said, who's who has not installed something using PIP as root? Should we break down some of the attack chain?

Yeah.

Was gonna say, like, so this is that's the you're you're kinda leading to that point, like, how did this even, like, happen? Right? And when you're when you're building something, especially now if you're using something like quad code to build something, chances are you're gonna say, hey, do this thing and it's gonna go off and pick libraries for you. Right? And bring that stuff in and like, do you have the chance to even like look at it before?

I mean, yeah, sure you do. But, I mean, I think so what what team PCP did is they they were able to compromise Trivy, which is a vulnerability scanner from Aqua Security, And they were able to overwrite version tags in the Trivy action GitHub action, so there was a flaw there. Right? And pointing trusted references to malicious commits. And so this was the second time apparently that Trivy had an issue too.

I guess in March 1, they had an issue that led to credential rotation. But yeah. And then so somebody else take the next part.

What is the Sure. So after after they hit Trivy, they additionally hit check marks with a with a similar attack targeting some of their GitHub actions and BS code extensions. And if you aren't familiar, CheckMarks makes code security scanning software. They've been around for for a while now. But what was even more interesting is what came after that, which is where LiteLLM enters the picture.

So how LiteLLM got compromised was actually a result of trivia getting compromised. Because LiteLLM CICD pipeline was using Tribi for scanning, and it pulled in a version that wasn't pinned. And that version that it pulled in was the compromised version of Tribi that it got hit by team PCP. And so LiteLLM CICD pipeline pulls in Tribi, kicks off the scanning, which then, resulted in one of the maintainers, I think, GitHub tokens, publishing tokens, getting compromised off of their system. And then so from there, the team PCP people were able to then insert malicious code into the LiteLLM, library itself, that then got distributed out.

And, Bronwen, I think you said the number earlier. What was it? It was, like, 90,000,000 downloads a month is about what LiteLLM?

According to an article by Endor Labs, which if you have the opportunity, look up Endor Labs, and the the title is team PCP isn't done. And it it has a a really nice breakdown, and they are claiming something like 95,000,000 downloads. I hate auto scroll.

Where is it?

Of course, the one line that I'm looking for, I can't find now. But LiteLL ah, LiteLLM alone. The LiteLLM library alone has over 95,000,000 downloads per month. Per month.

That's insane. So the, yeah, the potential impact of this is probably still an unknown, at this point. Right? Yeah. Because, I mean, this wasn't this is just within the last week, and so, that's a that's a huge scale of people who could potentially pull this in.

Now for what it's worth, my hats are off to the team PCP because they are they're not fooling around. They know exactly what they're doing. They are going after lots of of yummy credentials. They're harvesting SSH keys and environment variables, different credentials, crypto wallets. They've got lateral movement.

They've got persistence. They installed a system d backdoor so that they could do I and and nobody knows who they are. Nobody knows exactly who team PCP is, and they have managed

to They do now. Yeah. Well, so this is actually probably one of my favorite parts. Everything was encrypted and exfiltrated to models.litellm.cloud, which is not the legitimate litellm.ai. Hey, LiteLLM guys.

Go register those things. Well, it's probably too late now. Well, so it looks like that that was found by a researcher who was watching it wreak havoc on his machine. So that's kind of fun. I I'm kinda interested to see, like, how as a researcher, like, you would notice something like that.

And, I mean, I guess the crashing and stuff for sure. But then Yeah.

I guess it was I was trying to say my my understanding was, like, it was basically, like, utilizing all of his system memory suddenly, and he was like, hey. What's causing this? You know?

Yeah. Okay. That's good. So and then it looks like the makers of LiteLLM, Berry AI, got Mandy on the phone, which isn't cheap, and got them to come take a look. And so it turns out that if you're using the LiteLLM Cloud or the official Docker images, you weren't affected because of the the cloud and the Docker images using version pinning.

Which if you ever done any, like, Python and data science kinda AI type work, version pinning is your friend.

For sure. Oh, yeah. So you don't have to fight with dependency hell.

Yeah. Right. So so that, you know, that that's actually kinda one of the things that we'll talk about in a little bit about, like, well, what do you even do to combat this? But so it looks like team PCP kinda rose to prominence in 2025. That's year?

Yeah. Speculated ties to Lapsus, which I think if I'm recalling correctly is are those the folks who compromised some Microsoft stuff, Microsoft cloud stuff. And it looks like that so cloud and supply chain kind of stuff is their jam. Looks like they've also been attributed to compromising MPM packages, self propagating worm, using that kind of stuff. Like, I think was it the canister worm?

Yeah. Oh, that's kinda interesting. So apparently, the canister worm uses a a a an off protocol, Internet computer protocol for c two. What is ICP? Oh, okay.

ICP is c two? That's interesting. That

is interesting. We're gonna dub that the Juggalo protocol.

Yeah. Right? They also deployed an agent called OpenCall, which we're probably all familiar with. And then wow. So I think the takeaway here is, you know, everybody talks about prompt injection and model poisoning and the OWASP top 10 kind of, you know, like, you know, like, low hanging fruit kind of stuff.

But this the, you know, the actual compromise of this wasn't, you know, necessarily like an AI thing. Right? It's the same kind of supply chain stuff that we've all been using for a while. I mean, I would be willing to bet that they're at scale using AI to look for this kind of stuff, but it's not necessarily an AI flaw, which we've said many times on this show and to our clients that, I mean, it's not always the AI issues themselves you have to worry about. It's how you architect and build your solution.

And you know, I think the more custom that thing is, probably the more attack surface you kind of have. Like the flaws that we tend to see that are the worst are not they're kind of more traditional type flaws. And so it seems like that the folks that are making AI solutions haven't necessarily, like, learned the lessons from the past. Then also now with, you know, everybody being able to essentially code things.

Well, heck, even web developers haven't learned the lessons.

Oh, that's true.

How many times do we have no no data validation on a on a web app test? No. I mean, what what you're saying I I spoke recently to a bunch of CSOs, and one of the things that that I said, which apparently struck terror into the hearts of many of them, is that there's nothing that we're we're facing beforehand that AI is not going to amplify and accelerate. It's going to show us all the places where we're not doing the things that we should be doing, like version pinning, like data validation, like hardening our authentication processes, and now, you know, monitoring the supply chain for malicious code. Are we gonna have to start scanning every update for every library before installing it and running it?

I think, you know, one of the things that I would recommend is running as much as you can in Docker and in virtual private systems. Right? So that's how the model that I'm currently using is, yeah, I do use AI locally to build code. But, typically, when I run it, you know, it is going to be in Docker and and on a VPS, and it gives me a little bit more of a a safe feeling. I'm not sure that it would completely kept my keys from getting stolen, but it would have at least been just some keys and not like all the keys on my machine.

Right? So yeah.

Yeah. Yeah. So I guess that's a good segue into, you know, lessons here. So, you know, we talked about pinning your dependencies, certainly running things within Docker. You know, what are some other things we can do?

You know, credential rotation is important, not keeping static credentials around, especially for high value accounts such as maintainers on on repos. And then starting to treat some of these libraries and components as critical infrastructure, especially, you know, CICD actions that it's not, you know, it's not a given that that's that it's always gonna be safe because it's safe today, it's safe tomorrow.

Yeah. I mean, I and and, again, I just wanna reiterate that, you know, while this is related to AI and it is amplified, I think the whole, like, problem is amplified by this, you know, AI wave. This isn't necessarily new. I mean, Brian and I, I recall a specific red team we were on years ago. It was probably 2017 where a client we found, I think it was a Jenkins server that we could get on, and we were able to actually create builds.

And we were able to backdoor a custom build that gave us shell access on the box, and that was a root. And, well, we were in. Right? And so I don't think this is necessarily new, but I think, like, you know, you need to be very aware of what your developers are doing and the infrastructure and security surrounding them.

Yes. Yeah. Completely agree. I think that's a that's a great point.

Alright.

Any quasi thoughts?

You know, stay safe out there.

Yeah.

Yeah. Really. And and and check your dependencies before you download.

Yeah. Do a grep dash rail for LiteLLM in your home directory, maybe. I don't know. Yeah.

Alright. Well, I hope everyone enjoyed the episode. We'll see you next time, and keep on prompt