Vibe coding and supply chain security

[Darren] (0:03 - 0:22) One of the big worries here is that someone will be able to use AI to generate malicious packages with typos. Welcome to the DevOps Sauna, the podcast where we deep dive into the world of DevOps, platform engineering, security, and more as we explore the future of development. [Pinja] (0:22 - 0:32) Join us as we dive into the heart of DevOps, one story at a time. Whether you're a seasoned practitioner or only starting your DevOps journey, we're happy to welcome you into the DevOps Sauna. [Darren] (0:38 - 0:43) Welcome back to the DevOps Sauna. Once again, I'm joined by Pinja. How have you been, Pinja? [Pinja] (0:43 - 0:46) I've been very well, thank you, Darren. How about you? [Darren] (0:47 - 1:17) I've been okay. I saw a post on LinkedIn a while back. Actually, it was while I was sat at the Future of Software in London that I really wanted to talk about. We've had all these other topics to get through, so I'm glad we have time to get to this one. I was reading on LinkedIn a person who had created a platform as a service entirely using vibe coding and was surprised when things went problematic. [Pinja] (1:18 - 2:05) I think many of the good stories start with, I read something on LinkedIn and I wanted to talk about it. This is something that has come out before when we've had our conversations as I'm not a very technical person. I know the basics of software development as a meaning of the actual code writing. I know some of, let's say, writing something in Go. I took a dabble with that last summer. But I've always said that using AI would not make me a great developer. It would not make me a good developer. But now we're in a phase where we're talking about vibe coding itself. So this is a very interesting phase that I would not consider that we would get here so fast. But yeah, it did not go as well as was planned, perhaps with this example. [Darren] (2:05 - 2:28) Yeah, I think so. And as I've obviously mentioned before, being a security expert, I imagine a lot of people out there are thinking vibe coding would make me worry. And it does. I have to say, I think we should sit down and talk about why that is. And to do that, we have to understand the idea of software supply chain security. [Pinja] (2:28 - 2:34) So I guess we're going low back in history, and we're not just talking about 2025 or even the 2020s here. [Darren] (2:35 - 3:26) Nope, nope. I honestly don't know where this would start. So the idea of a software supply chain attack, it actually dates back, I suppose, to maybe even the whole Trojan War idea of, here is a horse full of soldiers. I don't know why the Trojans would have needed a horse, but that was the supply chain. They left, there was a horse, they brought it in, it went badly. But the idea is that no one writes anything, I can't say no one writes anything from scratch, but no one writes certain things from scratch. Things like authentication, some method libraries, things like making HTTP requests in Python, for example. There are libraries which exist to make this easier to simplify it, and they are things that everyone will use. [Pinja] (3:26 - 3:57) Yeah, and this reminds me of the meme that I saw many, many years ago, where somebody's asking, could you please explain all this? And they have a monstrous animal that is a mixture of many things, and basically just explaining where my code comes from. So it's part Reddit, it's part my colleague, what they've been doing. And as I say, we're taking so much, we're borrowing so much from everybody else. It's like, yes, there are still things that we write from scratch, but at the same time, we use quite a lot of libraries when we build our software. [Darren] (3:57 - 5:03) Yep. Honestly, like most of the stuff I've coded recently, because I'm not a coder anymore, I mostly do a little bit of scripting, but most of the stuff I do is just pulling different libraries together and making them interact with each other. So the things I write just act as integration layers, and this means that I expect the software that is written to be secure, and I import these dependencies into the software, and it comes out like, who's checking that code? We've had quite a number of examples recently, which we'll get to in a bit, but where code that has existed in even reputable sources has been hijacked in some way. So these third-party dependencies, they basically, the rise of this package-based ecosystem caused these issues where you had things like NPM, the pip repositories, RubyGems, they exist to serve packages to coders so they don't have to build these things, but, well, there have been some issues with naming, and I think that's the first thing we should discuss, the typosquatting. [Pinja] (5:04 - 5:21) Yeah, let's start from that, because then that gives us one of the new phenomena that we've been seeing. So if we look at some of the examples from, let's say, the last couple years, what has been happening, and looking at, for example, what has been happening with some packages and some exploits in there. [Darren] (5:21 - 6:10) Yep, for example, at some point there was a Python date util, it's like Python-date util, and then there was a Python underscore date util. Because there was no naming convention, then it was very easy to slip in a slightly differently named Python package that was then actually maliciously uploaded to the pip repository. So anyone who used a dash, an underscore instead of a dash, got the wrong program. And it's like, these things are increasingly common. When we were researching this episode, we went through some examples of this. We have quite a few of them here, and I'd be interested, Pinja, if you would be able to identify issues with these. For example, the next one we have is urllib versus urllib3. [Pinja] (6:10 - 6:44) That's a very good question, because if I were now to actually start writing code and started learning this myself, I might just think that urllib3 is just a version of urllib. And if I didn't have anybody to ask this, and I was just doing things by myself, I might fall into this trap, because we just added the three, right? But then again, we have some people who have been in this industry for a long time, but it is quite easy to make these mistakes and not pay attention to what the name of the package is. [Darren] (6:44 - 7:03) Yeah, I don't think it's down to experience level or knowledge or anything. A lot of these are just typos. For example, there was an express versus express with three s's. So if someone accidentally put a third s in, they ended up with a... It was, I think, credential harvesting. [Pinja] (7:04 - 7:21) Yeah, I was just thinking, because this is now in a popular NPM package, right? And it's a very frequent target. So how big of a problem would we, if we were thinking just like express versus express with multiple s's, how big of a problem would we be talking about here on the extent of it? [Darren] (7:21 - 8:21) That's a very good question. Like, probably depends heavily on the language being used. And as we know, I'm pretty sure the most extensively used language is still Python. So a lot of these examples, because mostly, whenever I have to script something, I use Python. So most of these examples are in Python. And if we're talking about those, we're seeing quite a widespread effect. And the whole point of it is, it's like most attacks these days. They're rarely targeted. They're volume-based attacks because they want a number of people to download these packages. They don't really care who, they just know that a number of people will make a typo or have an issue. There's another one that I liked, the colors versus color. And one was the British spelling, one was the American spelling. And it's like, these regional differences make problems for developers, especially if they're non-natives and don't really acknowledge the difference between the British English spelling and the American English spelling. [Pinja] (8:21 - 8:51) And that's for example, for myself, I am not a native speaker, not a native English speaker. So I would, I know that there's a difference. I have been taught that in my English class, but at the same time, the other one was in plural, right? So C-O-L-O-R-S versus the color with the U in there. So it is very easy, as you said before, since there is no naming convention here, maybe that, and maybe that is, am I using the British version? Am I using the American version? It is very easy to get confused with these ones. [Darren] (8:51 - 9:36) And it's even worse if you're actually switching languages, because both of these were actually valid. Like colors was the NPM package and color with a U, I believe, is the Python package. And I'm pretty sure they both do the same thing, but because obviously they're different packages for different languages and they're not beholden to any kind of naming convention, these now cause an issue there, which is, and this is an issue they can't do much to mitigate on their side. They name their packages. The space around them is not something they're subject to. And we've seen that in a number of ways. We see that in like URL squatting, where people will make a similar looking URL and use that for phishing attacks. So it's not just limited to packages. [Pinja] (9:37 - 9:51) Yeah. That was the example of .nvs versus .ev is again, one S and the one with extra S was actually a real malicious NPM package. So it was designed again, to steal secrets, right? [Darren] (9:52 - 10:38) Yeah. Yeah. The dot ends of it basically passes environmental variables, which is usually where people will keep authentication. So an extra S led to the leaking of a lot of secrets. And this is actually like this, this is not a new issue. This is an old issue that we're actually seeing migrate to the development space. This has been an issue with URLs for as long as they've been around. And there have actually been a few notable incidents recently. One of them, the famous solar winds backdoor from a trusted update. So this is like a code that we've trusted from a trusted vendor coming in. And this is also part of supply chain security because that vendor is part of your supply chain. [Pinja] (10:38 - 10:53) And then we had in 2018, we had the event stream where the maintainer handoff let actually into crypto stealing code. And so we have the malicious keeper stealing code injected in the new maintainer or to the new maintainer. [Darren] (10:53 - 11:48) Yep. Yep. There's a new maintainer, but hold of the code base. And just these are the things that it makes us wonder why we're not looking more at them, why we're not looking more at our supply chains. Um, like we're a DevOps company, this is a DevOps house. And from our perspective, we do, that's the whole point. The risk of all these things become magnified by DevOps. I'm not sure if I'm going to get in trouble for saying that DevOps makes it worse in this situation. But if you just think about it, if you have a coder sat in a room who built his code on his own machine and pushes it to a server without any DevOps, his attack surface is actually smaller than you have a hidden pushing code into Git, which is pulled into Jenkins, which is built and which is pushed into an artifact repository, which is then distributed by another pipeline to the server. We've magnified the risk by using CICD. [Pinja] (11:48 - 12:01) There's the automation part here that I'm constantly thinking like you're fetching everything automatically. You trust your dependencies. And that to me is the multiplication of, of the risk here. [Darren] (12:01 - 12:43) You're not just building the dependencies, but you're pulling everything needed to build the dependencies because Jenkins won't have that by default. Jenkins will be pulling out for whatever Gmake or whatever build system it needs, which then increases your supply chain's risk, which is, you know, you don't just have a Python package. You have the things required to build the Python package. You don't just have C code anymore. You have the things required to compile C, and then you have it being stored in a third location. So you end up with it stored in a third location and having a list of dependencies it might need when it installs. So you just, you keep getting this cascading problem. [Pinja] (12:44 - 13:32) Yeah. And as we discussed before, it is kind of like just moving it forward, right? And you like, as you said, you would like to trust that somebody has checked all of this and with the DevOps practices and the tooling, we are able to target some of these things. Let's say we take, yes, Nick, we have Sonatype and GitHub's got their Dependabot and they're able to help us and developers to identify the risky packages. But then again, we have human error available to us. And unfortunately not all the tools can take away human error. They're also built by humans so far at least. But then there's also the aspect of the novel attacks that of course come as fast as all these tools can identify new threats and start implementing them into their own flows. [Darren] (13:32 - 14:14) Yeah. Like most of these things are, they tend to be signature-based or knowledge-based. They fall into the same problem, don't get me wrong, Snyk and Sonatype are great tools and GitHub advanced security is so amazingly simple to install that I think everyone should have it by default. But they mostly work on things like signatures. They work in a reactionary way that security is stuck in. So having a proactive approach is in a lot of ways quite difficult. And unfortunately, the fact is now we're even still talking about the past because we're talking about five years ago, because now, as we know, our favorite subject on this podcast is occurring, and it's time to talk about AI. [Pinja] (14:15 - 14:37) So we're getting to the AI segment of this episode. It's not a 2025 episode of The Dawson without some discussion of AI. But it's just a fact that, as mentioned before, not even all the tools that are so up to date, they're state of the art, and not everything can keep up with what's going on. And we have some new attack surfaces here with AI. [Darren] (14:37 - 15:28) Yeah, we have a lot of new attacks to think about. So I think there is a tendency to think about AI security in terms of only the output, because that's what we've been talking about now. We've been talking about coding output, and we will actually get to that because there's quite a bit more to talk about on coding output and how we try to combat it. But there are actually new attack surfaces. So before we've had things like image registries to store built images, but now we have model registries. So this is an entire new thing. We have things like Hugging Face, which I think our internal Henry Derho described as, what was it? It was like GitHub for AI models. And it's like, yeah, people store their AI models there, and people will just download and use them without really analyzing what's gone into them. [Pinja] (15:28 - 15:54) It was basically like Stack Overflow, that what has been used for code previously, but now for AI models. And we already see this like just attacking the model weights, for example. Poisoning the model weights. And then we got prompt injections and vector vectors as well. So again, we're trying to trust what somebody else has built, but are we able to see what has actually gone in there? Do we trust that somebody has checked it? [Darren] (15:54 - 16:30) Yeah, that's the thing. If you are running a risk register, has it been updated to consider this even as an attack vector? It doesn't even have to be for a highly technical attack. If we talk about things like model poisoning or this kind of corruption, where someone could be inserting extremist comments into a self-improving chatbot and just utterly destroy your reputation, as we saw with Microsoft's chatbot many years ago, where after several hours of being exposed to the internet, it became wildly problematic. So we also need to talk about the data. [Pinja] (16:30 - 16:48) We do. And just like thinking of public training data and poisoning that, we can talk about it on a team level. If the team is getting poor quality requirements, the result is going to be poor quality. And the same goes with AI. If the training data is poor quality, you're going to get poor quality out of there as well. [Darren] (16:48 - 17:44) Yeah. And this is, I think, what people aren't really acknowledging enough about AI and the idea of open AI. Because I keep hearing things about open AI, and I don't mean open AI the company, I mean the prospect of AI being open and ethical and responsible. I don't know that anyone's opening up their training data. Like the training data, the data on which the entire model is founded, I don't think anyone's opening up on that. There may be people using public datasets, but people keep using this word openly, and I feel like they keep misusing it. I think LLAMA was described to be an open model, and yet we know Facebook downloaded or meta-downloaded 80 gigabytes worth of illegal books to put through their model. And so we have this consistent openness, which is not actually openness, it's just being used as a buzzword. [Pinja] (17:44 - 18:20) And this is something, I think, we already discussed in very brief when we had our Deep Seek episode, where we had our ethical colleague Henri Terho to talk about Deep Seek with us. And Deep Seek also claims that they are an open model. And again, have they published the training data is a good question. So here we go with that again. So we can see that we have these attacks. So for example, I mentioned Stack Overflow before. So anything can be manipulated on GitHub. We've talked about GitHub packages, for example, and flow has been taking over. So again, we have that channel available to be used for malicious actors. [Darren] (18:21 - 19:23) Yeah. And I guess the action point here would be to think about AI not just in terms of output. It is a tool, but it is a tool that has consumed data. It is a tool that requires a model. And if you have no control over that model and no knowledge of what data has gone into it, you have to acknowledge that as a risk and process that as a risk. And then that, of course, brings us to output, which I think is the bigger section here. I know we've been talking about this for a while, the idea of DevOps as guardrails on AI, because we're ending up with this thing called slop squatting, which I don't really like as a name, because it's basically typo squatting for AI. Because now, as we've talked about before, we've seen a huge increase in code volume, like the amount of code that comes out of people now is much higher because it's being written by AI in a fraction of the time. And that's a good thing. That's great. That frees them up for other pursuits, other objectives. They can do more with their time. [Pinja] (19:24 - 19:46) But there are some similarities here. If we talk about typo squatting that we talked about before, and we had a couple of examples here. So, model name confusion is very much related to what we just talked about. So, if we think of this as a trusted model, this is what I've heard everybody else use, but then there is something that looks like a typo, but is actually being used with a malicious intent in the background. [Darren] (19:46 - 20:56) Yep, definitely. And if you don't have a training case designed to catch things like that, it can be extremely difficult to really notice something like that before your AI model is out in production. And while it's not going to be the most technical or sophisticated of threat, the loss of reputation and trust if your AI chatbot starts responding in ways that are inappropriate is going to be extremely difficult. We've already seen the example where someone tricked a car manufacturer's chatbot to sell him a car for a Yes. Like that was a thing that happened. And again, this is why CICD around AI is so important because having test cases for these, AI is just the idea of prompts turning to output. So, it actually becomes very easy to test. You just need a dynamic set of test prompts to see the output compared to what you would expect for an output. And then I think one of our biggest threats though is the hallucinated packages. This is the slop squatting, the typo squatting. This is where we have to go back to the start and talk about vibe coding. [Pinja] (20:57 - 21:17) Yeah. And am I going to become a good developer by just quote marks, vibe coding and letting AI do my job? So, we're actually hoping that we do not see a decline in the quality of software because of this. And not just from the quality perspective, but from the security aspect as well. [Darren] (21:18 - 22:09) Yeah. And this brings it back to the LinkedIn post I wanted to talk about because it was about this guy who had created a platform as a service through vibe coding, but didn't really have the technical understanding to support that. And then when he hit issues, he had no idea where to start debugging. And this was something that he was already pushing live that he already had clients for. So, he was selling this and then an issue hit and immediately he had no idea where to turn. And I've tried coding with various LLMs and I think they're great tools, but sometimes when you tell it something's wrong and it does exactly the same thing over and over and over again, I can imagine how frustrated that guy would have been with trying to debug anything going wrong on a platform built entirely from vibe coding. [Pinja] (22:09 - 22:45) We do know that debugging on a bad day can be extremely frustrating on its own, even with human generated code, human written code. So, we do see this as an emerging threat. The landscape of software development is changing, that's for sure. And vibe coding is one of the threats that we see at the moment. But I see more and more people talking about this and not so much as not using AI tools at all for your code and your software. But please bear in mind that you do need to have the human element in there. [Darren] (22:46 - 23:40) Yeah, I think the idea of vibe coding is, first off, it's a weird name. I don't really get it. But we know in Effigo that a lot of people are going to be shifting over from the development and generation of code to the verification and validation of code. And if you're planning to do vibe coding, if you're going to institute some kind of vibe coding, if you're a person making a startup trying to do it like this, I won't say don't do it. I will say pay for GitHub Advanced Security. Get their instant one-click security options on your repos and use them. It's, I guess, kind of expensive to start with, but it will save you a lot of trouble in that way. And that's what I think we're really trying to say. Don't think that we're against coding with AI, but you need the tooling to support it. [Pinja] (23:41 - 24:01) Yeah. And if we think of the emerging threats, because we now talked a little bit about the past and the current landscape of supply chain attacks. So I guess the AI-generated malicious packages are one real threat. And you mentioned, Darren, previously the hallucinated packages. So that's something to keep our eye on also into the future as well, right? [Darren] (24:01 - 25:02) Yeah. I think that's actually going to end up scaling up too, because one of the big worries here is that someone will be able to use AI to generate malicious packages with typos. Right now they're being made at human speed, and we're actually seeing a lot of AI human speed attacks. If we think about malware, I think we're still seeing human-based attacks. We don't have good evidence of AI-based attacks yet, but they're coming and everyone knows they're coming. It's the same with packages. If someone can generate a hundred malicious packages in the time they used to generate one, we're going to see a massive increase in these typo-squatting, slop-squatting attacks, which is exactly why people need to use something like GitHub Advanced Security or Snyk or open-source tools like D-Track, dependency track, you could use to track these. And the requirement for those is no longer optional. It's something that I think is required going forward. [Pinja] (25:02 - 25:12) Yeah. And we think of the devil's quadrilles and just the safety net and like SLSAs, for example. Would that be something that would be sufficient from your perspective, Darren? [Darren] (25:13 - 25:48) Yeah, yeah. The idea is Salsa. It's like supply chain level security. And yeah, it's exactly what we're talking about now. It comes back to this idea that you trust dependencies like you trust your own code, but then for some reason you put your own code through all these security checks and just accept that your dependencies have gone through the same. And that's not always the case. So yeah, implementing Salsa, having proper guardrails around not only your AI output, but your AI input is kind of critical. [Pinja] (25:48 - 26:15) Yeah. Because to me, what really helped me understand this is the analogy of say somebody poisoned the flour at a bakery. So if you're now using that poisoned flour to make your goods and your products, everything is not just one individual loaf of bread that is going to be impacted, but everything else, everything you make with that flour, if you use that, let's call it poison data that your AI is using, is going to be tampered anyway. Anything that comes out of it. [Darren] (26:16 - 26:44) Yeah. And don't forget that one of the hallmarks of attacks is typically the ability to pivot. So to exploit a standing of where you are to get somewhere more advantageous. And imagine if your chat bot is randomly leaking company information to an attacker who has creatively poisoned it. This is a real scenario that we could be facing very soon and one that companies need to understand and prepare for. [Pinja] (26:44 - 26:52) Is there anything else in the defensive horizon now with the rise of AI and the scalability of these attacks that we should be looking at? [Darren] (26:52 - 27:47) Yeah. I think AI based anomaly detection is important. The big issue we've had in the past is you have to pass tons of data to get any real metrics. And a lot of this can be refined by being more selective with your data, you know, observability, that kind of thing. But now we can use things like NVIDIA Morpheus to pass massive amounts of data and make it useful in a human readable way. So I think we're going to see some advances on that front. But I think most of it is just developer education and this idea that I think we need to start calling zero trust DevOps because we've had zero trust insecurity for a while. But now that vibe coding is becoming more prevalent, now that people are using AIs on a daily basis, we need to get into the position of not trusting developer input in the pipelines. [Pinja] (27:48 - 27:53) Yeah. And I guess it also comes in developer education and just spreading the awareness of us. [Darren] (27:53 - 28:08) 100%. Yeah, people need to be aware. That's the same with all security things, though. It's like, as you mentioned, the human aspect is the problem. Making sure that people can identify the poisoned flower is the big issue if we want our bakery to run properly. [Pinja] (28:08 - 28:16) So I guess if we want to trust the dependencies, like with Terasa tools, we also need to be aware of who reviews that code, right? [Darren] (28:17 - 28:39) Yep, we do. And that's the, I think the key principle, and probably where we should end, is that we need to know who reviews the code that we trust, whether it's us or someone external. And so with that, thank you for joining me again, Pinyu, and we hope you join us next time in this honor. We'll now tell you a little bit about who we are. [Pinja] (28:39 - 28:43) I'm Pinja Kujala. I specialize in actual and portfolio management topics at Eficode. [Darren] (28:44 - 28:46) I'm Darren Richardson, security consultant at Eficode. [Pinja] (28:47 - 28:49) Thanks for tuning in. We'll catch you next time. [Darren] (28:49 - 28:55)