Talk: Improving Your DevOps Capabilities by Marko Klemetti, Eficode

Lauri (00:05):

Welcome to the DevOps Sauna. My name is Lauri and I am the chief marketing officer of Eficode. Not too long ago, we held a hugely popular two day, DevOps 2020 event. We had awesome speakers from around the world, telling stories about DevOps tools and culture for over a thousand people online. Since then, we have made these recordings available at the Eficode website and to the popularity of these speeches we have now made them also available in our podcast. You can find the links to the video recording and to the materials referred in the speeches and the show notes. We are also keen to feature topics that you find interesting in the area of DevOps in this podcast. Do let us see about you in our Twitter, Facebook or LinkedIn pages. This time we are listening to Marko Klemetti's speech. Marko is the chief technology officer of Eficode. Marko has joined Eficode more than six years ago. He has been doing a DevOps before it was called DevOps, and he has been exposed to many organizations on their maturity levels.

Lauri (01:12):

At the same time, he has been keeping himself busy with technology development by participating in a number of ventures. Marko's speech is about improving your DevOps capabilities. Let's hear Marko's talk. Shall we.

Marko (01:27):

So I said, having worked with DevOps or before that we called it Software Production Improvement for more than 10 years now, fourteen at Eficode. I've been working together with thousands of developers and also I've been happy to work with hundreds of different sized organizations in Europe and outside Europe as well. And, and as you've seen and heard already today from the presentations, one thing is sure DevOps is no longer a competitive advantage. So it's not something that you could seven years ago still go to and try to make your business fly faster. Instead, it's become an industry standard. And it's no longer question, if you should be investing in dev ops or not. You had the poll, we received somewhere around 600 answers to it. And more than 50% of the answers said that they are using Kubernetes. And big part, one fifth said that frequently. That already means that DevOps is definitely being used by the participants of this conference, but also globally.

Marko (02:45):

If we look at Dora, the DevOps research and assessment which was merged with Google, something like 18 months ago. Half of all organizations are capable of making deliveries at least once per week. And that's really just sounds weird. So if you peak high to elite performers from the last state of DevOps, 2019 report. You will quickly see that half of them are able to deliver into production at least once per week. Of course, for the elite performers that's not still much, but it's definitely something that should be looked into if you're not there yet. Some of the other interesting numbers is that if you peak from Dora or then the pop at similar state of DevOps reports, you will see that roughly 90% of the respondents, which are counted in tens of thousands are somewhere around 90% in the medium to elite or high performance category.

Marko (03:51):

Which means that there's automation is in place. There is some sort of continuous delivery or deployment automation already in place. They have been trying out content or since similar. And then if we look at, for example, CNCF the Cloud Native Computing Foundation, 84% of their respondents are running containers in production. And that's sounds really weird if I look back even two or three years ago, that just seems to didn't happen. And one more number from Datadog, which is an analytics company and also monitoring company. They made a state of serverless last year and their numbers is 50% of their AVS users are running also serverless, which is a huge number.

Marko (04:45):

So all of the new technologies within DevOps have been already, not only taken into use by the elite performers or are the first movers, but it has become matured. So why does it still fail? So as we're already reaching me, you've already seen that in countries, such as ours, where lockdowns are in place. Implementing DevOps is even more relevant than ever before because people cannot collaborate the same way. And the lot of us who are still working. For example, the IP business. If we look at how DevOps has been taken into use in organizations, I'm starting to hear the same as I heard years ago about the organization saying that, we say that we're doing agile, but actually we are nuts. And now that we've found a certain level of maturity in DevOps. I'm starting to hear, we're saying we're doing DevOps, but actually we're nuts.

Marko (05:51):

And usually the problems come from thinking of technological problems. As some of my favorites are we cannot automate the tests because it's not running web browser. It's mobile or IOT device, or what you're suggesting works on a startup or for a new product that we have lots of legacy. So it will not work here. Or our top management does not buy into it. So we've been doing some experimentation with containers and serverless, but it's not the direction for our organization. Or then yes, our competitors are doing this, but they've invested millions in it. And I'm actually really happy that we've had the presentations we've had this far, because for example, from the outcome of the H&M presentation, the things they've done and with the pace they've done, it has been such that it's actually reachable for everybody. And the same you will hear later today, you will hear Rasmus speaking from unity and they will also manage to do changes in a smooth way and really functioning way.

Marko (07:01):

I'm going to come back to that later. Also, one of my favorites is we are so special that it cannot be done here. And that's almost anywhere you go. You always, at some point hear that sentence. And honestly, this world is not so complex that it would be so special. I'll let you in on a secret, the best performing organizations fail in production, and that's not the joke anymore. So I decided to find some evidence for this and you've already seen. So you've heard stories from Facebook doing continuous deployment. You've heard from Google, you've not heard from H&M. And, and also you hear more during this conference, but the thing is the best performers in 2020 fail in production. Of course, they've invested in being able to fail in production. They've also even their teams and the development organization capabilities and permission to fail in production.

Marko (08:08):

But also what they've done is they've found ways to roll back. If a failure is rolled into production and that's for example, where Kubernetes or similar technologies come into play. So I was interested to see how the big players are able to do such deliveries on big, more difficult level. And I looked at Netflix and interestingly, if you look at Netflix release cycle for the mobile application, so I took Apple app store because it has more strict review. And the fact is that Netflix is constantly releasing new versions of their mobile application once a week, including the review, which is crazy. It's steadily, if you go and see the release notes, you'll see that they will be they're releasing new versions every week. So I went a bit further and I took the hundred top grossing mobile applications and their median for new releases is nine days.

Marko (09:14):

So if you now think that all of those releases from all of those providers are always perfect on bug free, you are wrong. You can get out of the Unicorn land and come back to here and see that everybody does mistakes. The way is how big mistakes are doing and how to recover from those mistakes. So my speak today, I'm going to be talking about three areas you can improve in your organization and the things I've peaked from the high-performance that you lead performance, but also things that are currently maturing as we're entering 2020. And these three areas, some of them are easier to implement. And some of them are a bit more difficult. The first and foremost, which is something that Rasmus from Unity taught, roughly one and a half years ago in our DevOps 2018, is building the production pipeline around your business targets.

Marko (10:12):

And as we already saw from Jako mentioned him, they are doing the same. So the point is whatever you are doing as a business connected to your production pipeline, it might be hard at first, but honesty, it doesn't say automated here anywhere. So the point is just to be able to track through or business targets, what you're aiming at new ventures, new products in the same pipeline, where you prove also all of the bugs and different change requests. That's the way you can actually start tracking or production as whole. Automation builds, reaches until you have self-organized teams. So once you've been able to find some way of starting the automation and as well as so most of you already have. Think of it more as bridges, like build bridges instead of fairies, then just having automation in place to make the heaters repetitive work easier, or removing that altogether.

Marko (11:21):

When you start building automation from the business requirements, you have to also find ways to stop the pipeline where you really do need some manual or brain work. And then you continue the automation. That's what the automated pipeline also means. So up until the point as what you heard from say the H and M presentation, until you have self-organized teams who can make the deployment decision themselves, I propose that you build the pipeline in such a way that you connect the different teams and different stakeholders together by the automation pipeline. I know the DevOps platform. So it's not a surprise that we also are working in the DevOps platform business with ethical roots. But also when you look at building the pipeline to which I'm going to come later today is when you have a DevOps platform in place, you're able to start like Legos.

Marko (12:26):

You start building a whole system where you can actually start tracking not only how your business requirements are walking through your pipeline, but how they actually behave in production and what things you need to take into consideration where building a product and also what kind of metrics you can pick from different phases of your production. Investing in a design system. I say here, investing in a design system and testing in natural language, you might wonder why combine these two. But the fact is that today's design systems and casting are the areas that actually bring people and the organization together. So in case you don't know what the design system is, design system is the voice and identity of an organization, but also a set of styles that represent that organization. So design system could start from very easy, having your colors and the look of your applications.

Marko (13:31):

It can then go all the way to having say CSS for styling or application, then all the way to HTML components and maybe even react components. And having a design system that evolves and also replicates throughout your certain area of applications. For certain organizations, it could be one design system is that standard replicated all that. For the design systems, you have to be able to combine the people who are working with the design and the development in such a way that you have a common language. One of my favorite examples is also coming from Australia. So Australian government has started their own design system. You can find it from designsystem.gov.au. And the design system they've done is completely open source. So their idea with their design system is that they've started a government of open source design system, which can be collaborated by anyone, which means that if you wanted to start collaborating to the Australian government design system, you would just open a pull request and someone there would then approve it or merge.

Marko (14:55):

But when you look at modern design systems as a whole, they're also part of them is the collaboration. So who maintains your design system? How do you manage having the design system parts developed? If you have a team that say working on a new kind of a date picker, and that team develops the date picker and the styles for the date picker, how do you pick them back to the design system that then can be distributed all over the organization so that everybody who needs that particular date picker can then easily use it. But also when you create the date picker it's work that you don't want to do again and again, which means that your organization has to find a way to collaborate in such a way that you work together until you find the way of working in self organized and also across that.

Marko (15:52):

And now when we combine the design system with testing. So if we look at testing, testing is the common language and could be the common language between stakeholders. In my example, I have robot framework using Selenium Library, and you could use whichever has thing that has natural language in it. The point is when you create test automation in natural language. You can have discussion over the test cases, which means that if you have the business requirements connected to your pipeline, and you write the test cases in the language understood by the business people, you already have an automated QA from the business requirements, all the way to verifying that it actually works and ready for release. And while you're building this testing and test automation, and you're writing the keywords, there is no reason why you wouldn't combine those also to the original design system.

Marko (16:58):

And actually now we could go back and look at the Australian government design system. They too have done the same. They're currently bribing the design system in such a way that you have the React components, but also the component tests as part of the design system. And in my opinion, this year or latest, the next year. People have started building design systems that actually include the pipeline, which could be, get help actions or Get lab or Jenkins or which server continuous integration or deployment pipeline. And also the infrastructure as code part of that same source or the design system comes from. The third area, validate business hypothesis in production. And this is actually really important part. It goes back all the way to saying failing in production, but validating business hypothesis in production is the way to go this year and in the future DevOps. It means that no way you can actually build quality assurance or automated testing or 100 percent verification that you can actually deliver the product bug free. But instead, try to find ways how you can go to production to a certain level of verification.

Marko (18:22):

One Of the ways to do that is enabling Canary releases. Many of the organizations actually already have a capability to release or do Canary releasing. Canary releasing in short means that you're releasing your production version to a very small amount of your traffic or users first. And then you look at how it performs and then if it performs well, then you can increase in steps or a big bunch, the amount of traffic you're directing to your new version. So the organizations who are struggling with building extensive QA systems, for example, this is definitely one way to go. So finding a way how you can get the modern tools, make releases that are smaller or directed to a smaller audience. The second part of validating business hypothesis in production is honestly, you cannot improve what you don't see.

Marko (19:27):

That also means that once you do the hypothesis testing in production or shipping, it doesn't have to be half made, but only a part of the features already for example. You don't know how it works, how your users will behave. What's the retention and how the approach from your users or customers are to the product until you've seen it in the production environment. But that also means that having the DevOps platform in place and understanding what happens both in the development phase and in the production phase is information that you have to collect and analyze in such a way that you actually understand what's going on.

Marko (20:13):

And what are the patterns emerging from there? So making your pipeline data driven is definitely one of the big trends that are coming. So to recap, what we've seen here, many of the organizations try to put the blame on technology. So as we already see, the technology is already mature enough for using the modern DevOps practices are bringing in the culture. But also I see that when you are fixing the technological problems, if you do it together in the organization, you're actually changed the culture at the same time. So the culture is usually the most important or most difficult part to change in an organization, but nothing will happen unless you build bridges with automation or then find ways of collaborating in other ways, during the development.

Marko (21:16):

As a reminder, best performing organizations fail in production. And honestly, nothing prevents you from failing in production as well. It's just more about how do you recover from that failure, or when do you know this, that there is actually a failure in production? So it's a level in between the risks you're currently taking and also the size of the delivery you are delivering. So of course the longer you delay your delivery, the bigger it grows and the bigger also the risk for a bigger failure gross. So usually failing in production means like the term shift left and also finding ways to fail often. So doing small releases often. The remedy from this presentation and what I see as the trends for DevOps to come from during this decade is that definitely threefold.

Marko (22:17):

The first one is a DevOps platform, meaning that you find a way to connect your organization through automation on collecting the information, what happens on each stage. The second hot topic is design systems. And once you've established one, how to maintain and grow the design system and how to connect all of your DevOps practices and tools into the design system. And of course it depends on you, how far you're ready to go with that. And then the third one is testing your hypothesis in production. All of the modern dev ops tools already enable you to do that. It's more of how brave you are and how you find the capability in your organization to do that. I've included a QR code here. You can scan it if you want, and today you will also see from our home site. We've released a DevOps for executive's guide, which has some of the things I've said today, but also some of the areas that we feel that are also very important, modern DevOps. And it's a guy you can skim through, or then dive deeper if you want to collaboration from our best people.

Lauri (23:35):

Interesting it was, wasn't it. You also heard Marko referring to the DevOps for executives guide from Eficode. You can find the link in the show notes, or look it up at our website, eficode.com. Next time around we are joined by Jakob Knutsson from H&M. In fact, Jakob's speech was mentioned by quite a few of our listeners and participants in the DevOps 2020 event. So it's definitely something to look forward. Jacob is a lead cloud architect at H&M and has extensive background in Microsoft technology. And is a leading expert on Azure Cloud, he has over 19 years of experience in implementation and transformation projects as a consultant. And now works as a lead cloud architect and is responsible for cloud adoption until then, stay safe and fair production.