Season finale: A journey through DevOps, open source, and AI

[Darren] (0:06 - 0:12) It all has to come back with making sure people feel listened to, people feel heard. [Marc] (0:21 - 0:53) Welcome to DevOps Sauna Season 4, the podcast where technology meets culture and security is the bridge that connects them. We are back in the sauna. Darren, I've really enjoyed Season 4 with you. How have you felt about our first season of the DevOps Sauna podcast together? [Darren] (0:54 - 1:11) I've enjoyed it. I was listening to you and Andy do the podcast for some time, so it was kind of an interesting shift to be put into Andy's seat and kind of dragging things a bit more towards security sometimes and having to rein it in on other times. It's generally been good. [Marc] (1:12 - 1:24) I've really enjoyed the gregarious conversation with you and the different points of view that we've been able to bring. And speaking of different points of view, this was Season 4. [Darren] (1:25 - 2:02) Yeah, so me and Marc have been having this ongoing discussion throughout the entire of Season IIII because when we start the podcast, we have to load up Squadcast and pick Season IIII. And we have four in Roman numerals or what Marc incorrectly thinks are Roman numerals because he has four I's. And I'd have maintained that four in Roman numerals is IV. And he's saying that for balance purposes on clocks, then to make it match better with the opposite side, the clock will have fourI's. And we've not really been able to solve this problem. [Marc] (2:02 - 3:01) I've enjoyed this one, as a watchmaker, over the years. This is something that I've kind of studied and I've toyed with. Big Ben is actually the biggest alternative to this with the IV for the number four. But there's a couple of different explanations. My favorite one lately, though, is that IV in Roman times was an abbreviation for Jupiter. So if you count one, two, three, four, it's kind of like one, two, three, God. But actually, I like the balance idea. And also, if you take the III and you look at the balance of creating the digits for clocks using while you're casting the metals out in order to make the digits, it actually balances quite a bit better as well. But we're thinking to actually just go with numbered episodes going forward. So if you see that on subsequent podcasts, then you'll know that we could not resolve how to name the podcast seasons. So we've moved to just using digits. [Darren] (3:02 - 3:23) Yeah. The irony, though, is we could have just gone for season five. I don't think we'd have had any discussion on the letter V. We'd have been just fine. But it is probably better because the seasons were an interesting way to split it. But I don't know how much purpose they serve. So I think it's good we've scrapped them and moved towards something a bit more podcast centric. [Marc] (3:23 - 4:01) Then we'd have the whole problem that Apple had with OS 10. People call it OS X. So then it would be DevOps Sauna Podcast Season V. So let's move right along then! So we're going to go through our episodes and talk about some of our favorite moments from there. We started out with a real heavyweight in the industry, the CEO of Control Plane, Andrew Martin. The episode was about navigating cloud security and automation. And there was a quote, Darren, that you pulled out. Yeah. The quote was, we used to say, if your sysadmins are playing solitaire, they're running a really good ship down there. [Darren] (4:01 - 4:24) It brings me back to the XKCD comic where there are two guys fencing on their office chairs and their boss asks, what are you doing? And it's, the code is compiling. So it's like when you're in the build process, you typically won't have a particularly high workload. Andrew Martin just came and talked us through a lot of really cool stuff about automation. [Marc] (4:25 - 5:33) And making sure that your Kubernetes is secure because it's not secure by default or out of the box. So if you're interested in more information there, then go back and have a listen to what Andy Martin said in our navigating cloud security and automation episode from the beginning of Season IIII. Now, we've talked a lot about platform engineering, and this is still going to be a very important topic going forward and talking about the pitfalls of platform engineering. And this was a quote that I pulled out that actually had originally come from a presentation on Ikea. They did a large transformation. We had a presentation on that in the DevOps Conference Copenhagen in 2022. And they said, culture is what you reward and what you punish, which I think has some interesting psychological safety elements to it as well. But it also kind of narrows down a little bit. Everybody talks about what is cultural change, and you can kind of look at your culture, and you can also see how maybe pathological statements like that one may have affected why people behave the way that they do in your company. [Darren] (5:34 - 6:10) Yeah, and I think some of the critical things we talked about there were the importance of commitment and investment, because it was very much focused around if you don't buy in and go in all the way, you're just going to end up with half engineering platform. You're going to end up with half a task done. And I think we had some really cool callbacks, because you were so enamoured of that presentation. And I'm kind of upset that I missed it, because it sounded like it was a good one. So I think that was one of the... This was actually the start of, I think, three different episodes on platform engineering we did, or maybe it was just two. [Marc] (6:11 - 6:20) Indeed, the next one was as well. And I think that you had a pro tip. So on getting started with platform engineering and where to start from scratch, what was your pro tip, Darren? [Darren] (6:21 - 6:26) My pro tip was in 2024, do not use Jenkins. [Marc] (6:27 - 7:18) And to our listeners out there that are happily using Jenkins today, if it's working for you, that's fine, but we wouldn't recommend starting a new project using Jenkins in the face of what's going on today. Because the quote from here is one of my favorite things about the bigger changes in platform engineering and moving to new tool chains, taking one of the big platforms into use, is where it can actually be a positive lever for cultural change, because now all of a sudden developers have this really clear 1.0 target. So this is something that we're getting a lot from our customers right now, the desire to defragment their tools and ways of working and to get into a single tool chain that can basically allow them to take time that developers were spending on their pet tools and move this into working on using their domain-specific knowledge for value creation. [Darren] (7:18 - 7:27) It is actually quite interesting because we took the quote from there that was about having a positive lever for cultural change. Let's listen to the quote from this episode. [Marc] (7:27 - 8:03) One of my favorite things about bigger changes in platform engineering, moving to new tool chains, taking one of the big platforms into use, things like this, is where it actually can be a positive lever for cultural change, because now all of a sudden developers have something like a 1.0 type of target that is very, very clear. Get your code over there and get it working, and that can provide an interesting amount of momentum towards making even more changes in the future, like developing new features or spinning off a part of the product or breaking down monoliths, things like that. [Darren] (8:03 - 8:44) That one matches into the next episode we did quite well, because we did an episode on managing change resistance, and essentially a situation we've both dealt with where you from a coaching perspective, me from a security perspective, trying to push change on people who might not be ready for it, might not be fully understanding of it, and how this is shaped our experience. Let's have a listen to it. You raised a knowledge worker as a driving force for this change, but it's also, in my opinion, one of the largest sources of change resistance. It's a knowledge worker who's set in their ways, who is very specifically looking at tooling as pets. [Marc] (8:44 - 9:20) The old pet tools, it's a big thing that we all experience, and we all even participate in. My pet tool at the moment is obviously Copilot, but getting really, really wrapped up in how one feels about certain parts of their environment. It used to be, when I was coming up in Unix and Linux days, that if someone else can use your shell, then you're clearly not a senior developer. Usually you come up and you just start doing normal things in someone else's shell, and it's absolutely haywire, kind of like Emacs or something like this. [Darren] (9:21 - 9:34) Yeah, I was just thinking, one thing I don't think we covered there is, in tooling bias, how would we convince 100 senior Linux users to switch away from Vim? That would have some considerable change resistance. [Marc] (9:35 - 9:48) I haven't seen the need to yet, so if you find a good reason for me to, then I'll consider it. I think if I tried, people would try to crucify me. And yeah, there's Copilot plugin for Vim as well. [Darren] (9:50 - 10:20) But we had a really good quote about this regarding knowledge workers. A knowledge worker is a driving force for this change, and that's kind of cool. But also, in my opinion, one of the largest sources of change resistance is that knowledge worker. It's a knowledge worker who's set in their ways and who is very specifically looking at tooling as pets. And that was, I think, the crux of the episode. We were looking into how to deal with the most problematic parts of DevOps, which is, unfortunately, at times, the people. [Marc] (10:21 - 11:21) And just to remind, the argument that I think is the most important one here is that we want to get people working on value creation using their domain-specific expertise. Taking care of their local tools isn't doing this. Sometimes, participating in a migration isn't doing that either. But oftentimes, when we see companies that are moving into single tool chains like GitLab and using those instead of maintaining lots of different tools and plugins and configurations and the connections between those, the people get a much more fulfilled life being able to feel good about creating something, not just about, oh, look, I managed a new auto rotation log for my Jenkins or something like this. So the next episode, Navigating Value Streams and Prioritizing Feedback. I would actually like you to read this one, Darren, because this is a quote I picked from you. [Darren] (11:22 - 11:56) So yeah, navigating value streams. I basically went into this podcast with very little experience on this subject. This is what it came out as. You did invite a security nerd to come and sit with you and have these discussions. So to me, it sounds a little bit like you're threat modeling a process rather than a set of tools. So you're kind of identifying your areas of weakness to try and minimize them. And that was the perspective that a full security nerd came up with from your musings on value stream mapping. [Marc] (11:56 - 12:38) It's genius. And it also highlights how important diversity is within your teams, within your organization, within the people that you talk to on a daily basis, maybe in the lunchroom or when you do your hobbies, that when people look at things through different lenses, they apply their context, their knowledge and expertise. So thinking about your process, having issues as a security person, threat model it and look at where the biggest threats are to efficient process execution. And that's a very, very nice way to frame value stream mapping in an organization. [Darren] (12:39 - 13:02) And I do think the diversity side is important, but it also kind of shows how similar the thought processes were, the actual decision-making processes. We'd have different labels and different names for them, but essentially we were kind of doing the same thing. But I do think we actually skipped one episode in our list. We actually have the incident response policy and practice that we jumped over. [Marc] (13:03 - 13:09) Yeah. I like the one here from you, Darren. Let's have a listen to it. Can we have a secure system? [Darren] (13:11 - 13:17) Not perfectly, no, but we can do our best. And that's all we're ever doing in security is what we can. [Marc] (13:17 - 13:30) And I think it's really important to understand that it's not just whether or not you are going to be hacked, it's what you do, how quickly, how well prepared you are for when you are hacked. [Darren] (13:30 - 14:07) Yeah. I think that message is important. I keep putting that message in blogs at the end of presentations because it's so easy to get bogged down in the negative when it comes to security and it's understandable. It's quite difficult to deal with if you have these situations and the better prepared you are, the better you will weather them. But there are thousands of victories daily that disappear in security. And that's why I just think it's important to say we're doing what we can and we're doing the best we can with what we have. All right. [Marc] (14:08 - 15:25) We both have a open source background. And if I were to maybe say a fascination or interest or enthusiasm about open source, so we absolutely had to do an open source versus commercial episode. And the thing that I said here that Darren quoted out was, and to me, nearly all the open source software that I was familiar with was built and maintained by software professionals working for a variety of different organizations and sometimes even collaborating across different organizations in the way that they contribute and maintain those components. And I think that many software managers, R&D leaders don't necessarily understand how important it is, the collaboration layer at open source projects that many of their professionals are participating in and that it's not open source is not just something that you take and you take from and don't contribute to. It's something that is a continuous collaboration in the software sphere that makes the important work that we're doing within our organization to provide value makes that work much easier because we're able to collaborate across organizations in the open source. [Darren] (15:26 - 16:05) We had a unique opportunity in that episode to kind of dismantle this myopic lens of commercialization and intellectual property that companies often view any kind of coding work as and just the power of these open source communities because of the tools they put together. And, you know, as you mentioned, we have open source backgrounds. We both use a lot of open source software and it's a message I'm pleased we were able to put out there. Sorry. I just, I read on our list the next episode we had the repatriation in DevOps one, which was our most sensible episode to date. [Marc] (16:05 - 16:22) Indeed. Indeed. And we did an April Fool's joke episode and I've never had so much fun talking tech. I just, I want to hear you say this again there. And so tell us what you said about the monoliths and repatriation in DevOps. [Darren] (16:22 - 17:05) Yeah. We were talking about switching away from microservice architecture and I said, just look at the pyramids. Monoliths are fun. Everyone likes building something that's huge. Like when I was playing with Lego as a kid, half the fun was just building the largest possible tower you could before it collapsed. And then when it collapsed, you rebuilt it to see if you could go any higher. And yeah, that that's maybe the tip of the iceberg of that episode. We went in a lot of interesting directions. I think at some point we were advocating the moving of all infrastructure into the basements of the people who wanted to use it to minimize latency. So we had some interesting diversions in that episode. [Marc] (17:06 - 17:45) And one of the, the neat things for me, when I think about it, we've had people around the, around the office and you know, some, some listeners here and there that I run into at conferences or on the street quoting this one. And I think that when you, when you do something like a satire kind of something, what you realize is there's people that believe a lot of these things. And most of the things that we said actually have some groundings in the beliefs that we hear while talking to various different types of parties around the world that, you know, there are companies that I know right now that are afraid of SaaS software for even basic things. And they would rather pay to run those things in the basement. [Darren] (17:46 - 18:03) Yeah. Yeah. I, I don't think you're wrong then. Well, I feel like there were a certain edge cases where we might actually run into places where it's sensible. Co-location can be a thing. And I don't know, it's just interesting to try and see things from the other side from time to time, even if we did have some fun with it. [Marc] (18:03 - 19:17) Very true. Very true. Our next episode, navigating the AI open source minefield, kind of dovetailing right out of the previous episode on open source. We were really pleased to have our good friend, Amanda Brock on this one. Let's have a listen to our favorite quote from this episode. Open source is a lot of different things, depending on who you are and how you've come to it. But at its heart, there is the legal and licensing requirement, which is that the software is not only made open, but it's distributed on a license, which complies with the open source definition. And reminding ourselves and our listeners that, you know, open source means it's free to use by anyone for any purpose. And that is something that people are trying to change. They're trying to reframe what open source means and add these kind of commercial riders to it. That if, well, if you have more than this many users or something, then you have to come back and negotiate a license for us, which means it's no longer open source. Or if you're building some type of application that goes against the values of that project, that you're not allowed to use the software from that project, that means it's no longer open source. [Darren] (19:17 - 19:55) Yeah. And I think Amanda came and added this, this very interesting perspective that neither of us had because of her background in legal situations where she kind of introduced this to us from a business point of view, exactly as you're saying this kind of diluted shift between something being described as open and something being concretely defined as open source and how so many tech systems are trying to float being open as the same as being open source and just all the difficulties that came from that and this kind of dishonest representation. [Marc] (19:56 - 20:51) Indeed. Our next episode, I appreciate Darren, the way that you framed this one for me. You said, if you're only going to listen to one episode, make it this one. And essentially prior to the summer holiday, I was happy and productive, but still, I recognized that I had a fairly high stress level and we had an episode about it and talked about different tools that I have employed or been taught or prescribed to others. And it was quite cathartic to get some of this out as well. And if you listen to one episode, as Darren recommends, listen to the one that we did on maintaining psychological safety. Let's have a listen to our favorite quote from this episode. Although I understand that I am in a stress condition and I've been for some number of days, my work is brilliant. I love the work I do and the people I work with and I'm not overworked by any means. [Darren] (20:52 - 21:05) Yeah, I think it's important. And I don't think there's much to say about it. It's something that you need to listen to in full. So I think we just leave out of that and move on to the next one because it's worth the All right. [Marc] (21:05 - 21:47) Our next one, innovation and infrastructure with another great friend of ours, Cheryl Hung. And she said. I think the challenge of AI from an infrastructure point of view is scale. A lot of the things that we've worked on are not really ready for the complexity, the storage and the computational complexity that AI brings. So perhaps in some bigger organizations that already exist, but that's going to filter down to the medium and visit the smaller organizations as there's more and more demand to solve various problems using AI. So I mean, that's my that's my overall thoughts on how that's changing. [Darren] (21:48 - 22:43) Yeah, we talked a lot there about the infrastructural requirements for AI, because as we all know, we're dealing with the dawn of AI now because we finally have the computing power to deal with it and companies finding innovative ways to implement and keep those systems on the rails without going wildly over their AWS bills, for example, is important. So yeah, we talked about quite a few things there, and I feel like we made some headway in a topic we're going to be hearing a lot about for years to come. And then I think with the next episode, we had a brainwave, which we started to activate and will hopefully continue to do so in the future, where we just decided to round up some DevOps news from the previous month and looked into the things that happened, discussed them, discussed our thoughts on them. [Marc] (22:43 - 23:40) It's really good to, you know, one of the reasons that I listen to podcasts like this and one of the rewards of doing a podcast like this is to be able to get other people's opinions on not only what's important, but, you know, to also get help understanding what's going on out there. And this is a great practice we're going to continue. We will comb the news for you and we will give you our perspectives on what the important things are and what we see is going on in the world. And speaking of something that's going on and really important, our next episode was on accessibility. Let's have a listen to it. I think it's still about the lack of awareness. It's just something that they have not thought of for some reason, because there's a lot of information available, a lot of good training and materials online, but I guess there's still lack of awareness. [Darren] (23:40 - 24:28) Yeah, this was an episode that was in the pipeline for quite some time because we thought it was an important topic and very much wanted to get it out. And then we had some delays that don't really matter, but it was an important topic that doesn't get airtime. And there was this very cool quote from Annika, who is in our accessibility team about there still being a lack of awareness. And it went, I think there's still a lack of awareness. It's just something that they have not thought about for some reason, because there's a lot of information available, a lot of good training materials online, but there is still a lack of awareness. And from a security perspective, that was just like an arrow directly to the heart when Annika said that. It was like, yes, I know exactly what you're going through. I know that exact problem. Absolutely. [Marc] (24:28 - 24:52) And the thing that's really I want to highlight here is accessibility is not for a minority of your business. Accessibility is for everyone. And it's something that is not only mandated in many legal jurisdictions and the internet being a global thing, if you have an online presence, then you need to consider the accessibility for your global audience. [Darren] (24:53 - 25:30) But talking of mandates, I think next we move on to my favorite episode of what we've done so far, which was earlier in the year, the EU decided to pass the first legislation about AI and its potential abuses in various areas. So I was really excited to talk about that one. Let's listen to the quote from this episode. The EU has had some success in litigating these things before, but they've mostly done so in physical space. This is an attempt to manage these things in digital space, which might be a bit more complicated. [Marc] (25:31 - 26:57) Regulation does serve humanity, and I believe that it is necessary, especially if you can get it right. And getting it right is not always easy, but in many cases, this is the best systems we have. So I applaud the EU for following through and looking closely at how AI is going to affect us in the future. And I also hope that this scenario will continue to improve. One of the big important things that we talk about from our work as DevOps practitioners is the AI safety net. And we had a couple of guests, Henry Terho and Kalle Mäkelä from Eficode, who are working in our AI department. And what was said there is... Yeah, I guess this is the thing now, that everybody wants to move really fast and break things with AI. But when you end up breaking things with AI, stuff just evolves so quickly or goes into some direction so quickly that the risks there are quite big on it. But then also, if you get it right, you get a huge amount of benefits from that. People outside put AI into customer support, content creation, language translation, localization, education, cybersecurity, a lot of the different fields. But I like very much this move fast and break things is something we've heard for a long time, came from Silicon Valley. And it's really important, though, to realize that AI moves even faster. And if we're moving fast with AI and breaking things. [Darren] (26:57 - 27:52) Yeah, it kind of redefines the rules. It's actually something we see. And I'm going to drag it back to security again. But we currently know that AI is not being applied widely because reaction speeds are still human. And once we start seeing AI reaction speeds, that's going to be a game changer. And we understand that insecurity. But I think a lot of developers are kind of just pushing for breaking things, breaking boundaries, building fast, and damn the consequences, which is... It's an interesting way to move, but that's the position DevOps has. It has the ability to put the rails on the side of these rockets to make sure they don't fly off the track. And I think AI is learning now with the AI acting with the safety nets that they do need those safety protocols. We have gone past the point of reckless abandon, I would say. [Marc] (27:52 - 28:42) And just as a reminder, some ideas behind the guardrails or the safety net that DevOps can provide for AI is how you test and how you deploy safely to make sure that developers are comfortable making changes, even if AI has been involved, having clear policies so that developers understand that they are still ultimately responsible for the code that they ship, even if they've been using AI tools to create it. Now, our next episode on developer joy with our friend Sven Peters from Atlassian. He said... Maybe also it's our own fault because we call it a development pipeline and we think just like it is like a pipeline, right? You throw in an idea in one end and get the working software out at the other end, right? And all the time in between we need to optimize, but it's never like that. [Darren] (28:42 - 29:09) Yeah, I think it was very curious to have Sven join us and give us a rundown of development from a position that at least I as very much a nerd didn't really consider and considering it from the human point of view and how to get that feeling of joy back to the developers and how mechanical it can be to make DevOps and kind of suck the joy out of creating. [Marc] (29:09 - 30:11) This is something that's evolved for me a lot even since this episode. I have a customer, I have a small production system that wrote with a colleague and now I've been maintaining. So I'm handling production software for the first time in many, many, many years and I realize how much energy and joy I get from the coding and creating experience on a daily basis and all of the other things that are necessary in order to make that code work, how we do the documentation, how my deployments work, all of these basic DevOps types of things are fun and it's great to get it running in production. Of course, that's the ultimate reward, but there's a lot of overhead that's not necessarily as joyous as creating and seeing my work running in production. So we go through a lot of these kinds of things in our episode with Sven, but I don't really know how the next episode came about, Darren. I had an interesting weekend and then we had a conversation about it and then it turned into an episode. [Darren] (30:12 - 30:59) I think it's one of those where we just kind of came into this and pressed record and it was like, yeah, okay, we're talking about a time you were repairing a bike and trying to dismantle that and take whatever lessons we could from it. Let's have a listen to our favorite quote from this episode. I think there's a lesson to be learned in museum security and that they don't try to keep people out if what literature I've read is to be believed, but they try and keep people in after they have made an attempt. And obviously we can't keep people in, but we have to look further inside than just this perimeter because that's where our threats sit. That's what people haven't quite understood is that 90% of attacks, I think, are a result of user error. [Marc] (30:59 - 31:51) It was a lot of fun. And I think one of the things that I remember from this episode is that when we look at whatever we're doing and we're able to apply experience, force, advantage from multiple directions, then we're able to get a better result than we do if we just kind of come in hacking. One of the things here was there's something that you said that there's a lesson to be learned in museum security because they don't try to keep people out if what literature I've read is to be believed, but they try and keep people in after they've made an attempt. And obviously we can't keep people in, but we have to look further inside than just the perimeter because that's where our threats sit. That's what people haven't quite understood is that 90% of attacks, I think, are a result of user error. [Darren] (31:51 - 32:32) Yeah. What happened was you managed to pivot the idea of repairing a bike into this defense in depth model in cyber security, where if you just look at the perimeter, then nothing is really going to help if a threat comes from inside, which we're starting to learn that a lot of them do. So yeah, we had some interesting life lessons coming from that one and reflecting and kind of lending credence to the things we talked about with regard to tooling and security. So it was honestly, I really liked that episode. It was fun because I don't think either of us had any idea where it was going when we started. [Marc] (32:33 - 33:50) The next episode reminds me, you know, another reason that we do this and, you know, one of the greatest things that I've learned in life is that, you know, the thing that matters the most is what you can do with other people. And we had on an Eficodian, Mr. Dan Plumley, GitLab's newest GitLab champion, to talk about his work, his evolution from being a software enthusiast all the way to becoming, you know, an elite member of the GitLab champion club. And this is one for, I think, all of us to listen to because he paints himself so beautifully as, you know, just someone who loves what he does, has found his path in life. And, you know, we all suffer from things like an imposter syndrome or, you know, not knowing what we want to do or how we want to do it or what we want to become when we grow up. And Dan's story is just a really beautiful story of how he has become successful in this role. Let's have a listen to our favorite quote from this episode. You're not just having a look at the issue that you're on there or the code file that you're on there. You can ask GitLab, Duo, tell me about this whole branch and it'll do so. [Darren] (33:50 - 34:30) Yeah. And we have these kind of core tenets here at Eficode about changing the world, learning every day, completing each other. And I think most importantly here, taking pride in results and just being able to have a colleague on here that's doing that, that we can lift him up and say, this is exactly what we're doing and where we want to be going was just an extremely cool, cool thing that we could do. Just so pleased with being able to get that out. Excellent. What do we expect next season, Darren? Well, I think we're expecting not to be having seasons anymore so we can put the Roman numerals discussion behind us and finally focus on the work. Thank you, Darren. [Marc] (34:30 - 34:48) Thank you all for listening and we'll see you back in the sauna next time. We'll now tell you a little bit about who we are. Hi, I'm Marc Dillon, lead consultant at Eficode in the advisory and coaching team, and I specialize in enterprise transformations. [Darren] (34:48 - 34:54) Hey, I'm Darren Richardson, security architect at Eficode and I work to ensure the security of our managed services offerings. [Marc] (34:55 - 35:02)