Securing the future: SBOMs, AI, and seamless DevSecOps

[Priyanka] It's a multi-fold approach, right? Like when you think about shift left, one is in terms of what you can do, right? And the second part is also sort of equipping your team to have the right knowledge. [Marc] Welcome to the DevOps Sauna, the podcast where we dive deep into the world where software, security and platform engineering converge with the principles of DevOps. [Darren] Enjoy some steam and let's forge some new ideas together. [Marc] We're back in the sauna. This is a DevOps Conference Scandinavia pregame podcast. A reminder that we're going to be in Copenhagen on November the 5th and Stockholm November the 7th with the DevOps Conference live in Scandinavia. And today I'm really honored and pleased to have Priyanka Syal in the sauna with us today. Hello, Priyanka. Hi, how's it going? It's great. I'm so happy to have you and I'm always happy to have my dear cohort, Mr. Darren Richardson. Hello, Darren. Afternoon, Mark. Correct again, Darren. So Priyanka, it's so cool to have you in the sauna and you're going to be at the DevOps Conference talking about implementing DevSecOps, navigating regulations and enhancing security. Really important topic, really on trend right now. How did you come to this talk? Like, what's the background that has brought you to us? [Priyanka] Yeah, I think, I mean, I started as a software developer and then as in my journey, I've gotten also hands-on in terms of infrastructure. And quite recently at BCG X, one of my longest running projects was in healthcare. And there I got this input into how do you have DevOps? And then there comes a security part in a regulated environment. So previously I'd done quite a lot of work in DevOps as such, having CI/CD pipelines and having automated deployments. But I think I touched a lot on security in terms of regulations in that project. And it got me really interested on how do we create systems or how do we have security in this shift left way, right? How do we incorporate it in our process in a software development life cycle? [Marc] I think it's really interesting how we kind of get led onto a path. And sometimes it's by nature, we have talents in a certain area, or sometimes it's just the path we're on kind of nurtures us into something. And then we find ourselves with a specialty like DevSecOps. By the way, Darren, I think you have an internal or the internal Eficode definition of DevSecOps. [Darren] Yeah, it's not something I think we've ever really made public because I think it's kind of controversial, but we have this internal discussion saying that there is no DevSecOps, that DevOps requires security, otherwise you're doing it wrong. What do you think to that? [Priyanka] I think now with the experience that I've had, I think I sort of agree. I think, again, it's very context dependent, right? You need to figure out what's the right amount of security. There is a definition based on the domain you're working in, the context you're working in, what sort of data you're working with. But I do agree there has to be some amount of security that needs to be incorporated as a part of your DevOps process. [Darren] But I think domain is actually one of the key things there because you mentioned earlier, you were working with healthcare devices and at least maybe you've noticed the same, but I've noticed in the world of embedded systems, security is often 20 years behind where security should be. [Priyanka] I think one of the things that we were working on at that time was having a shift in what, how the client was working. And so we were incorporating much more like software solutions in their product. And I think with that, though, of course, I agree there were these elements of security requirements, for example, that I saw, which I would feel that, okay, why are they here? But then still, I think they did a great job to sort of try and incorporate the more latest software definition of security, right? Because when we think about medical devices, we think of, as you said, embedded security in an embedded systems environment. But I think when you are like as today, I think almost everyone is working towards incorporating much more software. You don't want for every new feature, you don't want to give a right. You want to incorporate, you want to give your clients, your customers basically features as fast as possible. And in that environment, you have to go towards having a much more software idea of, you have to basically transform your product more towards software. And now I think they also had to shift their idea of security. So that's an interesting thing that you brought up. And I think because we are also dealing with data, right? It's all sensitive data. So where you're storing it, how you're storing it. And on top of that, if you think about the other regulations that come in when you're thinking about data, for example, GDPR, that also comes in. So I think the companies are trying to move towards a much more modern approach to security. Of course, it'll take time, but I think they are on track or they are trying. [Darren] I would certainly hope so, because we've actually seen some changes. I think in the last couple of years, there was a novel attack on a medical imaging device from a nearby university that was actually embedding malicious code into the scanned images. So we're starting to see a lot more evolution in terms of the complexity of attacks. So hopefully companies are starting to keep up with those. [Priyanka] Yeah, we did have things like tooling for container, analyzing the containers. We had SCA, we had SAST in the pipeline. So I think they were doing those things, right? They were moving towards doing those things. And of course, moving like as an entire, for example, as a huge organization to shift that idea of security, to have shift left, to have this idea of zero trust, it takes time to follow those principles. But I think they did, for example, establish a security team. They did go through this. They did try to establish a process and it's an iterative process, right? Journey. It's not, you can't figure it out in a day. So you do something, you figure out, okay, this is working, this is not working. And then you go back to the drawing board and you add more steps or you shift how you're doing things. [Marc] I've seen shift left kind of almost as a thrown over the wall type of idea to some development teams like, okay, we're going to shift left security now. And then kind of the pointy boss walks away. So it's nice to talk about the iterative approach and yeah, it's just work. You try some things and then you see what works and you keep going. But do you have any more tips for shift left or a little bit of experience that you'd like to share? Because one of the things that I advocate for culturally is shift left and security and many things. It's about giving developers the tools and the space to be able to learn, understand, and take responsibility for those things. It's not just a, now you're responsible and that's it. [Priyanka] I think it's a multi-fold approach, right? Like when you think about shift left, one is in terms of what you can do, right? And in that, it breaks down to different things like having a process and then for example, having a tooling. So for example, in terms of process, I would say, let's say you're designing a new feature. Are you doing threat modeling when you're designing it? Are you putting that when you're architecting any new feature? And then in terms of tooling, I would go to like, what sort of tooling do you have in your pipelines? Do you have SCA, SAST? I think it goes there. And the second part that you talked about is also sort of equipping your team to have the right knowledge, right? Like sometimes we are developers and we don't have that security mindset. And how do you make sure that everyone on your team is trained enough or has that information? Maybe you have like a V team where you have security or you have like security champion and everything, which is making sure that these sort of processes are being followed. So I think it's a mindset approach. And then from top, if you look at it, do you have security as a functional requirement, right? Is that a part of every feature that you're developing or a non-functional requirement, for example. So, and then how are you tracking it, right? Like, is it like, are you tracking it in whatever, for example, Azure DevOps or Jira or any way that you have requirements, wherever you're listing your requirements. So it goes from top down and maybe also to recognize the gaps that your organization might have in some of these capabilities, right? Maybe you, as of now, don't have someone who is great at security that maybe you hire, you bring in someone. Maybe if you're dealing with compliance data or sensitive data, then you bring in some experts there and you involve them in every part of your process, right? You don't just involve them at some time and then it's left off, right? You sort of make sure that in that time of maintenance, you have these teams working in and making sure that you're up to date with, let's say regulations or compliance or any new things that are coming in. [Darren] Yeah. I think it comes down to the engagement, like the tool set. If you build, it's no different from any part of DevOps, really. If you build a great tool set that doesn't actually require people to use it or doesn't take them into account, they're going to ignore it. And it's the same with security. So I like how much you leaned on engagement. [Priyanka] Yeah, I think with tools, it has to be a transformation also of the mindset, right? As a, maybe there is something that's lacking, right? Not everyone. So as a developer, for example, if you can think about, can you also, for example, have compliance in code, right? And, or you could think about, oh, this is something that we are not doing right. So it has to come from all sort of all roles. And it shouldn't just be only one person is responsible for it. And that requires upscaling as well. So I think that's also something that's important, you know, to have that shift in the mindset. [Marc] You said something and then you changed it. And I want to go back because I think you said something super important that maybe it was a Freudian slip. I don't know. So I'm just recently talking with some different customers. And when I was coming up, there was product owners and product managers. And when agile came around in the late nineties, early two thousands, and then we tried this business product owner and technical product owner thing for a while. And now there's these business area leads and technical area leads. We have an F-code and some other places. And, you know, one of the core tenants of this is that product managers don't care about non-functional requirements. All right. And then you said security is a functional requirement. And I said, hallelujah,<laughs> maybe that's the way that we can get the business to care about security. I think that that is a genius innovation that came out here today. [Priyanka] No, I agree. I think, again, it depends on how a company is slicing their work, right? Sometimes I've seen, for example, performance can be a functional requirement or a non-functional requirement. Now, it really depends on like how you are, like what you're dealing with. I think that, for example, in a regulated industry, security is a functional requirement, right? But for example, in other industries, which it might just, it's still important to have it at least as a non-functional requirement, right? That needs to be the bottom line. So for regulated, I think it has to be definitely a functional requirement. But non-regulated, again, putting this might be over-engineering at some point, because again, how do you define security comes into picture, and maybe they don't have bandwidth for it. But still, it has to at least be a non-functional requirement. [Marc] Speaking of requirements and of regulation, I know that there is, you've done some work and some discussion about SBOPs. And this to me is another one of those things that was kind of quite natural. I grew up with a lot of embedded stuff, and you had a bill of materials at the factory, and you had a software bill of materials included there. And it was just rather kind of natural. And then when we go around many companies now to deal with regulatory environments or the US executive order on cybersecurity, things like this, people are now kind of panicking on getting their SBOPs together and figuring out where all their dependencies come from, things that you would think should be really important anyway to know what's in your software. So what's your take on SBOPs? Are people doing it right? How does it change with AI today? What's your take there? [Priyanka] I would sort of have that analogy, right? Like when you go to a supermarket and you're buying something to eat, do you look at the ingredients? Do you want to look at the ingredients? Or do you just want to have everything that's out there? I think it's, again, right? Like for example, here in Germany, you have like a rating on the food, like A, B, C, D, E. And I think it depends on what the ingredient list is. So again, it goes back to the fact that what are you following? I would like to have CRS for. I would like to know what I'm having or what I'm consuming. And there's a very practical reason for it, right? Like tomorrow, if there's a, and we all know, like in terms of, for example, Log4j, what happened there. So if you have a third party dependency, and you do not know that there's a risk that has happened, and your customers don't know about it, then you, it's just, it's like a black box you're creating, not just for your customers, but also for yourself, because you're not going to be able to track what went wrong, when it goes wrong. So I think for me, SBOM is super important. It's like a hygiene issue, right? You need to have it out there. And also it speaks a lot about the culture that you have, right? Are you transparent and open about what you're building? And yeah, I think that it's absolute must at this point. [Darren] It's becoming, yeah, a lot more important too, with the advent of AI, because now, not only are we seeing this requirement for SBOM, like the software bill of materials, but we're also coming into this new concept of having this kind of training data bill of materials, where all of these AI tools are going to need to be able to replicate their, or note their training sets to ensure that they can trace output to do things like combat bias. So now the requirement for it is, people aren't even doing it properly at the moment with software, and the requirement for it is about to explode with AI. So I think now is probably a good time to get on top of it, if people aren't doing it. [Priyanka] Definitely. And you've brought a very important point, right? It's not just the model that you're using, it's also the training data that you're using. And for example, even for now, I know a lot of companies or organizations are using chat GPD, for example, the enterprise version. Do they know all the training data that has been used and is giving them the responses that they are getting? Not a lot of people know that. And also, if you think about it, I think at some point it's going to be a lot, it's also going to fall back to like an ethics issue, right? And a security and a privacy issue, because when companies have to show what sort of data they're training their models on, they have to be very wary of where the data is coming from. And if it has some sort of consent to it, if it has, if it is correctly sort of anonymized, and also how are they acquiring that data? And I think that's going to be a huge challenge for a lot of companies out there. [Marc] This is almost a tangent, but I kind of thought of this earlier today. So I'll try it with you too. Am I the only one that's thinking about version control for co-pilot logs, co-pilot chat logs? Because when I started customizing my shell in the old days, and the first thing I would go do is maximize the amount of history and just keep everything that I possibly could in my shell history. And then I can always grab for things that I've done, you know, a year ago and redo that. And then being able to have, I had to, I was forced to open a new chat in co-pilot, because there is a bug that you ask a simple question at, it starts to show you the answer in co-pilot chat, and then it takes it away and says that this is, you know, filtered based upon restricted content or something. I'm like, what? I just asked, you know, how to refactor a function and it gives me this. So one of the cures for that is you start a new chat. And then I realized, uh-oh, now I've got more than one chat open in the same workspace. Should I start adding these to Git or not? [Priyanka] That's an interesting thing you've brought up. I don't know. [Darren] Like what, what prompts are you using? [Marc] Yeah. [Darren] Yeah. I think it's, I mean, maybe Git's not the answer, but some kind of like ChatGBT just implemented the shared memory across conversations. So that that's the feature that co-pilot is probably going to follow along with in the next few weeks. So I don't think you have to worry about the issue for long. [Marc] Yeah. And that bothers me too, because oftentimes I open a new one in order to ask the same question and get a new random response. [Priyanka] I agree. I agree because I, uh, I'm sometimes not happy with the responses that I'm getting. So I'm like, oh, can you give me an alternative way? Can you give me an alternative way trying to get something which makes sense? Yeah. It's interesting. I don't know. How would this evolve? I also think it's such a, we're in that phase where it's such a feedback loop, right? Like between the users and it's, it's so new on even, even for us, right? Like how we are using it. It's so new. And there are some places where I think it's great to use. And sometimes it's just garbage, right? And you're just like, what did I just see? This does not make sense. Uh, sometimes, for example, if I have tried to generate code from chat GPT, it's garbage code. I'm sorry to say it. It doesn't work. If I, if I copy pasted as is in my IDE, it's just red, it doesn't compile. And, uh, and if I look at it, it looks okay. So I feel it, uh, yeah, it's still a feedback loop and to expect that this will, again, it's like an iterative process, right? We'll get to a solution, but I think more, more than the prompts, I'm more concerned about the data. I'm more concerned where my, the data is coming from and how is it being used? And how is even my data being used? Right. [Darren] I do think you're right about the feedback loop, but I think we can also describe it as a faulty feedback loop. If you've ever been caught in one of those infinite cycles where you try and correct chat GPT and it puts out exactly the same thing over and over and over again, and doesn't understand that it has done something wrong, but it continues to do so. [Marc] Yep. I have an analogy for this, which is like when you're vacuuming the floor and there's that thing that the vacuum won't pick up. So you pick it up and you look at it and then you throw it back on the floor and you try to vacuum it again and it doesn't work. So then you pick it up and what do you do? You like crumple it or you break it into pieces and then you still can't vacuum it. It's just, it's never ending cycle sometimes. And then you come back later and it gives you exactly what you want. [Priyanka] Yeah. No, sometimes you just have to throw the trash out yourself. [Marc] Yeah, sure. Exactly. [Priyanka] You have to start a new window and you're like, okay, this is not working. [Darren] There's one topic I kind of want to touch on because there's this idea in security that I think a lot of people have, which is that they're too small to be affected by anything, by attacks, by regulations. And this is something I run into on occasion where people just think I'm too small to be attacked. How viable do you think that approach is in 2024? [Priyanka] I think if you are developing code from scratch where you're not using any third party library, you're just doing your own thing in silos and it's not going to be out there. Sure. But of course, no small company is doing that. We all have so many dependencies to begin with. So I think one part of it is like, sure, you might not be susceptible to an attack, but what you're using might be susceptible to an attack, making it your responsibility to make sure that you have process in place to make sure that whatever you're using is actually secure. And the other part is that when you say you're not dealing with any regulations, I don't think that's possible. I don't think that's possible in today's world. I don't see, because you have to deal with data, right? And there is data regulations for something. I genuinely don't think it's possible. [Darren] I think the point of a startup tends to be to make the minimal code to make two existing bits of code work together in a novel way. That's probably the most succinct way a startup works. And then it does come back to the whole software, the supply chain attacks that people just don't consider and still aren't considering. [Priyanka] But even in that, right, when you're making two pieces of code work and you're a startup and you're doing that, I think still it's when you say, again, when you describe security, I think it goes back to, you have to define for yourself what security. I think right now, some of the things are very low-hanging fruits. Having some sort of steps in your pipelines are very low-hanging fruits. You can easily do that. There are also open-source solutions for it, and you don't need to really go and have a very specialized security expert to do that. You can just have some checks in your pipelines and make sure you have some container scanning happening in case you're using containers and you're making sure you're protected from any malware being injected. I think it's important that people have that at least small-hanging fruits defined and that addressed. And as you grow, you go into a little more depth and you try to have maybe a security team or some sort of champion who makes sure that some of the requirements are met. But I certainly think it's becoming really easy now to have some sort of, to have DevOps with implied security. With having that DevSecOps and DevOps equated in some way, I think we are there in some way. You just have to know what steps to incorporate. [Darren] Yeah. You mentioned something very quick in there about the low-hanging fruit of, and you also mentioned container scanning. I fully agree there. Scanning is so seamless most of the time. If you're doing it manually, you can launch Trivia with a one-liner to scan your containers. In GitHub Advanced Security, it's like pressing one button. So how important do you think the seamlessness is to security in DevOps tooling? [Priyanka] Describe seamlessness. What do you mean by seamlessness? [Darren] In a way that it wouldn't basically slow down or interfere with the developers, so that it requires the minimum cognitive overhead, so that it's just something that sits there on the pipelines and doesn't interfere with their work. [Priyanka] I think that's super important. I think as a developer, you don't want to be thinking about other 10 steps you need to do except building, right? As a developer, anyway, I feel our roles have changed. As you grow more and more senior, you're not just coding, right? You're doing 10 other things. You're managing stakeholders, you're figuring out sprint plans, you're figuring out the tasks you have to do, you're figuring out integrations. There's a lot more to do. You're also maybe managing infrastructure, you're managing testing. So there's a lot more things that have evolved in our jobs now. So making this process seamless, I think is really vital and having adoption. Because if it's a pain point for me, I won't do it, right? And every time, for example, I have to build it and then run even that one line, it's an overhead. And that's why I feel like having these steps as a part of your pipeline, or maybe even in an automated build process in your local, like having the scripts that you're just running all the time, that builds and creates this image and scans for it is something that can be done and is super helpful. But having that seamless integration, I think is very, very important. It's a good point that you brought up. And also, I feel, again, I would add on to seamlessness, I would also say accountability, right? Like if you are telling someone that do this extra step, then how are you tracking it? So not just seamless, there needs to be a way of having a reporting attached to it or having some sort of tracking attached to it to know that this has been done. So I would say that both of those things are important, or like having a checklist or in some way ensuring that this is done. [Marc] Fantastic. Are you excited about the conference? [Priyanka] Yes, I am. [Marc] We are so excited to get to meet you in person and get to hear your talk on implementing DevSecOps, navigating regulations and enhancing security in Copenhagen on November the 5th and Stockholm November 7th. Priyanka, thank you so much for this conversation. It has been wonderful to talk with you. And I'm more excited than ever to get to meet you and hear you talk in person. [Priyanka] Thanks a lot. It's been wonderful to talk with both of you. And I really, really enjoyed this conversation. [Marc] And I'm really pleased that I'll put this at the end for our best listeners, that this was your first podcast, wasn't it? [Priyanka] Yeah, it was. I had quite fun. [Marc] All right. I think you did really well. Thank you. All right. Thank you, Priyanka. [Priyanka] Thanks a lot, Mark. [Marc] Thank you once again, Darren. It's always a pleasure, Marc. Okay. We'll see you next time back in the sauna, and we'll see you at the DevOps Conference in Copenhagen and Stockholm. See you. We'll now give our guest an opportunity to introduce himself and tell you a little bit about who we are. [Priyanka] Hi, I'm Priyanka Syal, and I currently work as a Senior Engineer in PCGx. Prior to this, I worked in different industries across different companies. I worked at Microsoft, EMC, and a couple of startups. I enjoy building and maintaining infrastructure of large-scale applications. And outside of work, I do improv. I enjoy going to the movies and festivals whenever I can. [Marc] We'll now tell you a little bit about who we are. Hi, I'm Marc Dillon, Lead Consultant at Eficode in the Advisory and Coaching Team, and I specialize in Enterprise Transformations. [Darren] Hey, I'm Darren Richardson, Security Architect at Eficode, and I work to ensure the security of our managed services offerings. [Marc]