Discussion with Xray: How culture builds quality?

Sérgio (0:09): Whenever we think about developing software, you're trying to see what is the impact of what we are delivering. And we need to be able to track that in a more or less automated way, also, I like that we need to have some time for ourselves for human thinking and to also take advantage of the experimentation. So for example, we have Xray exploratory hub. It won't replace you as a human, for exploring the system, but it's there to help you out so you don't lose your focus while you are exploring the system. The tool is not there to really replace you. It's like there to augment your own capabilities.

Lauri (0:55): Hello, and welcome to DevOps Sauna. The DEVOPS Conference is coming again on March 8th and 9th, and you are invited to join our event. To build the excitement to the DevOps event of the year. We have invited some exciting people to join our podcast and share a bit of backstory to the themes we will be covering. Today we have Sofia Neto, head of ecosystem experience, and Sérgio Freire, head of solution architecture and testing advocacy, both from Xray. From Eficode, Szilard Szell, a DevOps consultant joins our conversation. We talk about how culture builds quality, why is it hard to build high-performing teams, and what characteristics should the teams be looking for in their tools. Let's tune in. So we are here with Sofia, Sérgio, and Szilard. Welcome, everybody to the DevOps Sauna podcast.

Sofia (1:51): Hello.

Sérgio (1:52): Hello, Lapa.

Szilard (1:53): Hello, Lapa.

Lauri (1:54): Nice to have you here, we have an interesting conversation ahead of us, I was looking into the questions and I was trying to have some statistics, have some insights that can be shared across all of us. And I looked up Puppet 2021 state of DevOps report, the whole big theme about the state of DevOps report from Puppet was that many companies have become better over their last three years or so, the amount of companies who are very high state of DevOps evolution has almost doubled, but still almost eight in 10 companies are still what's called stuck in the middle. And my question to you is, when we talk about DevOps, what do you think is the biggest challenge that the organization face? And what could be the reasons behind this inability to develop organizations?

Sofia (2:48): Well, first of all, I think that we have to think a little bit about what DevOps means. I think there's a lot of confusion still in the industry about DevOps, you see often, for example, people that call themselves DevOps engineers or they create DevOps teams. That's a lot of confusion. And I think that's one of the problems right now. And there's a challenge for the companies and organizations to understand what means DevOps, if you don't understand what it means and how it impacts the organization, you're likely to fail towards the goals that you want to achieve.

Sofia (3:16): I think that everyone wants to increase the deployment frequency, wants to decrease the lead time for changes, wants to decrease the time that it's needed to restore a system when it's needed, and also wants to decrease the change fail rate. There's not a clear understanding what it means to do this, or how can you achieve this. What I think is the biggest challenge right now, it's the mindset because really, tools are there, are available, and what really needs to change, it's the culture and the way people work. And it's more about practices and the way you work and behave. That's the real change that needs to happen because there are some kinds of practices that you need to embrace that most of the companies are not aware of that, and they need to do this kind of understanding and this kind of thinking. And knowing that DevOps, it's about culture, it's not a job title or not a team. It's about culture, and you need to understand what needs to change in your culture in order to succeed.

Lauri (4:56): I was looking in the sum of the numbers there and there were maybe two numbers that stood out for me, one was 21%, saying that culture discourages risk. And then the other one was 18% report, that the fast-flow optimization is not a priority. Any thoughts around that, Szilard?

Szilard (5:19): Yes, I think that discouraging risk and the culture of actually not to take risk is very much in line with what Sofia mentioned about the culture and the behavior because many of the companies had for years processes in place to reduce risk, right? And then to avoid risk. And that was the most important that we never fail. First- time right behavior was so much in there and punishment for failure was so much in there that I would say very hard to change that culture and to change the way how people think about failure, and how they think about taking a risk needs to come from management and high level leadership support. And this needs a different leadership style as well. We say we celebrate failure until we can learn from it, right? We see failure as a learning opportunity and improvement opportunity getting better.

Szilard (6:16): But if the base culture of the organization is about not to have risk to avoid them for any cost, then it's really hard to break. Now the other thing is the flow. And I think the flow is similar because if you have a fast flow, that means you have a lot of changes, and that's exactly what we want with DevOps; to introduce more frequent changes from development into operations and bringing value to our clients and end-users. But that means change, and change is a risk. So I would say this comes back to the risk-taking part as well. If you're not stepping away from that, then we will build big administrative manual decision points in the process making sure everything is low risk. And with that, we will create giant batches. And the big batch is actually against the flow. So I would say that these two have the same root in the risk avoidance culture and the behavior.

Sérgio (7:17): I always remember from my past life, because I actually have several lives, only my best. And one of them was on the Telco area. And I remember that we have those big releases that could only happen like twice per year or once per year. And we did lots of them checks before being able to make the release, and we would schedule the release for the weekend. Not on Fridays. Fridays, it was not possible at all. It was like a non-existing day. And well, deployments would happen on Saturdays. Actually, we were researching a R&D company. And we have Telcos as customers and the Telcos would ask us,"Well, are you sure that this will all go fine? Because this cannot fail." And that mindset of trying to avoid the risk at all costs, I would say it leads nowhere because in the end, we are going to have problems. But my question is how we are going to learn with them? Do we have the toolings and the practices in place to easily recover from those problems? And in another project that I worked for, it was a job board. In another life, of course, it was a job board. And actually, we're migrating from a pass solution platform as a service solution into our own development platform. And that migration was from place A to place B.

Sérgio (9:08): And you couldn't move the other way around. And I remember it very well because it happened around Christmas. So imagine my how Christmas was back then. But we had to make sure that we have the mechanisms to deal with failure. And for that, we need to have, let's say several things such as monitoring in place, we have to define, let's say, what are the minimums for the service to be considered okay and how can we deal with other problems later on after we sleep for some hours? Because for sure, we want to get everything right in this migration process during the first phase. So, dealing with risk, I would say is very, very important and we need to find ways for us to be able to be agile enough to push the changes and the fixes, and then have necessary means to have a working product. And well, eventually, the product will always have problems, we know that testing is not something that assures that you got the problem without any bugs whatsoever. There will always be problems. But in the end, we need to deal with risk. So we need to figure out what's valuable, ensure that what is valuable gets working best, and if it doesn't work best, we need to have some thoughts and some tools in order to figure out workarounds to deal with that.

Szilard (10:43): I very much agree with Sergio on this. In the whole, this quality assurance, where I'm routed from, we are trying to reduce the probability of delivering the faulted production, reduce the probability, but then the other aspects of DevOps, when we are able to reduce mean time to recovery, and be able to come together and solve the problem quickly, those are all about reducing the impact of a fault or a failure in production. So this DevOps are actually providing terms for both of these aspects of a risk. And you need to be good on both of the areas that you overly reduce the risk. So actually, it's strange, you need to reduce the risks, so your culture can live with now the reduced risks and then go on for the more frequent changes and the more frequent flow. So really, this is a journey I'm consent with.

Lauri (11:39): Yeah, you said that the changes need to be small and they need to be basically turning around very quickly. And I remember there was one programming language, I think it was Rust, and it had a very, very strange syntax, in one specific part of the way how the programming language was written. And I asked why on earth is that syntax so strange? And the answer was that when you make changes to the code, if you have to add a line on the code, you will not have to touch the other lines. So when you look at what has changed, then the syntax allows you to only have those lines highlighted, that actually has changed. So the coding language was helping a developer to identify where exactly that changes. So if something is wrong, then it's easier to spot and then again, it basically improves your cycle time. I suppose coding language isn't the first thing where organizations start when they think about addressing the culture. So maybe there's something else but when we look at adopting a DevOps culture, then what should be the first concern? Any thoughts about that?

Sofia (12:56): Well, so the first thing, I think it seems to be the easiest one, but it's often forgotten. So I think the first one company must be concerned on making the work visible and understanding exactly the flow because only when you've achieved the understanding of all your system, when it starts, when it ends, when it stops, whatever it happens, only when you have that kind visibility, you are able to build around that. So yeah, and making the work of visible is also super important in terms of creating trust. And if you are willing to fail and if you are willing to take risks, you need to trust your system. That's something that it's easily forgotten. And I've worked with some companies trying to adopt DevOps, and they forgot about this. So this is something that I think that if a company hasn't developed a good system to make the work visible, and it can be like a physical board, it can be a Trello, it can be Jira, whatever. What really is important is that everyone knows exactly how the system works, how the workflows through the organization. And that's one of the things I think that it must be the first concern because if you are willing to take these risks, you need to trust your system. So you need to understand how it works, and how you can recover. And I think that's the first step.

Lauri (14:43): Make work visible, right? Interesting you say that. I remember when we were looking into some of the projects in 2005 that we should undertake to make work mobile. We had an R&D unit in Tokyo, and they were using this kind of physical Kanban board. And basically, they were telling us that this is what you need to bring mobile. It used to be physical then, and now it is digital, but that was their way of making work visible.

Szilard (15:14): I very much see that what you say. So, I'm very much keen on Lean and Lean management and even visited couple of factories working with Just In Time and Toyota Production System. And they still now like big big big factories, car manufacturing factories, using the paper on the wall, the best visualization and the most flexible. And they don't play around the JIRA, they have all the tickets on the wall, they have the elements on the wall. And what I really loved like, was it a plastic molding company producing bricks- brick toys, you might imagine which one I'm talking about. But they put always every day the failed pieces on the machine so everyone could see how does it look today, when we have a bug, when we have a failure, something failed, it was so you could go there and touch it and see exactly why this piece is wrong. And it was on display, you understood why it is wrong. And honestly, this is what I'm sometimes missing when we deal with Jira or we deal with Trello, or anything else like the feel of I go there, I touch it, I see it, and we communicate. But now with the remote work, of course, we need to have all the digitalized boards and everything like that. But I agree, make things visual, and of course make work transparent and visible.

Sérgio (16:41): I think this leads us back to the first question where many companies still struggle to figure out what DevOps means. I remember when I started researching about it. And because we think it's something like physical and it's not really something physical, but to also answer and taking the answers from Sofia and Szilard. I think that visibility is one of the key aspects indeed. I see teams nowadays using all sorts of tools, even not just Jira, Trello, they use visual tools, sometimes even middle boards, if that's what they can facilitate. But I think there's still some space in that area for having something more fully integrated with what is happening in production. So I think that having a board is really important because whenever we don't have some way to visualize it, what happens and I remember what happened in the past in some of my team is that we have to ask, "What's happening? Were you able to do that? What's the status of that? And it's a lot of time lost in questions when you could see exactly. If you have, let's say, the proper tool, you could see exactly the status of it. And having, let's say, an open channel for conversation. I'm laughing because I remember that there are so many companies that struggle having, let's say, a real collaboration platform, you need to have a real collaboration platform where people are free and open to speak. Because if they are open to speak, and they are not afraid of speaking that can be the first thing in order to solve, let's say, some ongoing issue or some ongoing problem. And sometimes people are afraid. Sometimes they don't have the tools in order to establish a proper collaboration. So I will also work out on the more soft skills on the team skills that will foster collaboration.

Szilard (19:10): And maybe coming back on this one I very much like what you said, Sérgio, and my big thing always blinking how to adopt DevOps culture, and we need psychological safety, right? So people need to be able to speak up.

Sérgio (19:26): Thanks for the word, I was missing it.

Szilard (19:28): Yes, they need to collaborate. But for collaboration, that means that you need to be able to ask the stupid questions without punishment, being able to fail again, like make mistakes and learn from it, being tolerated for that. But also sharing information, knowing that if you share what you know, you won't become less important, right? Unfortunately, I see, and I'm working on large-scale DevOps transformation. I see that many times people do not want to share their knowledge because they are afraid, they will be irrelevant later on. And I very much believe that there is a need for the leadership. It's a different leadership style, what is needed. And it's not like after one day training, you just change the leadership style and with that you change the team's style, right? So, it's a very long and hard process and a very soft skill-based journey. One more thing to add there is the common vision, DevOps, and the journey towards DevOps is a giant change for big companies specifically, is a big change. Why do we do the change? What is that we want to achieve with this change? What is it for me? What is it for every layer of the organization? And what I can do for it? So sometimes they say, "Yeah, we want more frequent delivery." And then whenever you're talked with, they say, "No, we don't want more frequent delivery." So make sure you have a goal, and you communicate the goal more frequently, and you stick to that. And every decision, every communication, every direction, is supporting that goal.

Sérgio (21:11): If I may interrupt, you're saying one aspect, I would say, that makes completely sense because I've seen this in the past whenever trying to make working with a team to making a bit more Agile, because they were liking really, really, really waterfall world. And for the manager, back then, the manager didn't really see like major benefit of doing, let's say, the Agile transformation because, well, he knew that a release would need to happen on day 10. So he wouldn't care what will take for people to get the result because their release will need to come on day 10. So, in the end, we have to understand, what are all the team works? What are the, let's say, the blockers, which from my perspective, most of the times they're not really technical blockers because technical or blockers, we can easily more or less easily overcome them. Most of the times we have more management blockers. But we need to fill out the pains at different levels. And probably we'll talk more a bit about that later on. But in order for us to be able to provide value. And to sell, let's say the DevOps, the Agile transformation, you need to understand what are the things that matters to all the stakeholders involved in the software delivery or in the value delivery chain, and that requires a bit of conversation before we actually start doing some stuff with the team.

Lauri (22:59): It strikes me that there are still organizations or individuals who think knowledge is power. And especially when combined this, let's say, lack of shared vision or lack of communication of sharing the vision, I would imagine that if you combine them, it's quite impossible to do the best job, it's quite impossible to serve your customers the best possible way. How can you reach a high-quality execution of work if you're holding back the best possible implementation, or the best possible practices? It sounds to me that something more is important than then doing the right thing. So when we talk about the quality, or in other words, doing the best job, where does that fall in this DevOps picture?

Sofia (23:47): Well, I really enjoy hearing you talking about the safety that needs to be in place for us to have a real DevOps culture. Because only when you have this kind of trust, and this is the most important thing, I think, is that you need to trust your team, you need to trust your system. And you need to trust that if you are going to fail, you are not being punished by that. So this kind of safety is super important. And I think it's really what relates to quality because really, if you are able to trust your system, you're going to build quality and you're going to go faster.

Sofia (24:39): So you are going to improve the collaboration because you know that you can share the information with others, you know that you can fail. And this is something that it really impacts the quality of your service or the way you serve your customers because you are all together creating the best possible solution. And I think that testers have a great role here. Because they are really helping the organization to increase the trust because they are testing, they're experimenting, they are trying to learn more about the system and making sure that it delivers value to the customer. And when you combine this vision, I think that you are really building a true DevOps culture because you are learning from the failures and testers can help you with that.

Sofia (25:38): So you are taking these testing activities in learning about the system and experimenting, and making sure that the value that what you want to deliver is actual delivery. And also, here, there's something that sometimes in the DevOps culture, there's a questions about what's the role of the tester. Because do we need testers themselves? Because we need to build quality in or not? Of course, we need them because they are super important to coach the team about quality. They're super important about helping the team to learn more about the system. And I think that's a question that I would like to hear, or Sérgio, I'd like your opinion about that. And Szilard, I think that you have something to say, right?

Szilard (26:30): Yeah, sorry, I'm very much eager. And I'm a tester by heart; 21 years in in the testing area, and quality assurance, but I would be very careful to mix to talk about testers, when we talk about quality, right? It's not equal, it's not the same. quality is done. How quality is done? First of all, we build quality, then we try to capture missing quality. And then if we delivered missing quality, then we try to solve it, right? So it's all that needs to be in place. And for me, in DevOps, the Quality Assurance is not only testing, absolutely not, if we can avoid a mistake, a misunderstanding by having, for example, behavior-driven development applied, having better understanding of what do we build for the end-user, but I would even go to the left more applying design thinking and going into service design more to understand what the clients want, what the end-user wants, and even they don't know what they want, analyzing them until we know better, how we can serve them better.

Szilard (27:43): And that understanding upfront is more important than capturing usability issues or flows in the service flow later. So I believe that quality is extremely important to make sure that everybody is there, everybody is responsible for quality, then we just say that that's a buzzword, how? But when we give the visibility, we give the tools we give the techniques, as well, and the processes and the communication and everything to have that to build the quality to realize on quality is missing. That's very important. Then on the other hand, I totally agree testers can help a lot to get feedback, right? The second way of DevOps to amplify feedback, if you look into that one, of course testers can have a lot of support there.

Szilard (28:35): And I very much say that when a tester is asking on a on a requirement identification workshop or anyway and what if not, that's my favorite question: and what if not? This is what you want, this is how it should work, and what if not? So coming back there and the tester just asking that question in a place where in avoided for older organization, they might not even be able to show up. I think that's extremely important. So let's talk about quality assistance as a new term in DevOps and let's make sure that everybody is assisting each other on having a good quality, but yes, of course, testers are important. We are testers.

Lauri (29:20): It sounds to me that what needs to be done is very clear, but why is it then so difficult to get the team together to do it?

Szilard (29:29): Maybe just on thinking on this one. So what I realized in during the transformation, so it is extremely hard. We are stuck in the current state and we are working hard, the organization is working hard to deliver on the promises that have been made. At the same time, we try to renew ourselves. So for example, we bring in new behaviors that okay, so testers that you go and talk with product managers and product owners and be available upfront. But at the same time, you should do your job, which is now actually doing whatever automating tests or exploratory testing, accounting whatever you do. So for the change, somehow you need to have the time to do the change, right? And you need to be able to apply these new approaches, learn them, apply them, fail in them, relearn, improve, and go on. And I think that somehow, I see that is very hard. So serving the legacy, as well as working on the future. And probably this is where many of the organization stuck, that's my view.

Sofia (30:39): Well, I guess for this, we can all remember one famous quote from Einstein, that "it's easier to change an atom than to change prejudice". And I think he's exactly what you're saying. Because you work in a certain way and you don't have time to just stop and start behaving in the in the way that you want. And that's something that it's really hard to do, us as human beings. And for example, I can share with you a personal example, I wanted to start doing more exercise, but I was not able to do it. And I really want it. I truly wanted to do more exercise. So what I did was I started to go to the office by bike. So I implemented a different behavior in my daily routine. And this is an example. Of course, it's not about DevOps. But it's really hard to change your behavior and go after what do you want, and maybe by start doing some small changes, for example, if you instead of not sharing the information, using a Confluence page, or using whatever tool that you want, but instead of not sharing that, putting that information available to everyone, these kind of small changes can really have an impact, and really change the culture in the end.

Sérgio (32:20): If I may just add small bits on this. So I was hearing what you, said, and I was remembering the loss-aversion bias. Because even if we see some gains, we really don't like to let things fall away. So what happens in some teams, is that they sometimes they build out their modules, their stuff, the way that they work, the way that they communicate. And sometimes it's really hard to change that because people don't really like to lose things lose what, especially if they build out something. And once again, this leads us to the mindset, we need to be open to try out things and to let things go away, which is really hard. It's not easy to do.

Lauri (33:23): It's Lauri, again, building quality right into your software development is a necessity. To learn how it works and how to get there, we've recently released a new Continuous Quality Assurance guide that will give you the foundational understanding around the area. Whether you work in management, development or elsewhere, this guide talks to you about test automation, test design, test metrics, test environments, test data, and the future of continuous quality assurance. You can find the link to the guide in the show notes. Now, let's get back to our show.

Lauri (34:01): Once in a while we have related to let's say Trello or Jira, and things like that. But what I am not hearing you to talk is overall, the importance of tools. Talk me through this tools aspect of cultural change, and tools aspect of building high performing teams like where would you start, and what behavior would you like those tools to drive?

Sofia (34:33): First, when you think about tools, I think you have to be in mind that the fact that you are wearing Cristiano Ronaldo soccer shoes, you're not going to be as good as he is. So that's the first thing that you must think when you select a tool because I've worked many years as helping other organizations implementing tools. And most of them were thinking that, "Okay, I'm going to buy this tool. And the next day, I'm going to be Agile," or "I'm going to be DevOps." Or whatever. And that's the first thing that you need to think when you think about tools, but of course, tools help. So it's really hard to write a book without a pen, or without a paper, that's hard. So you really need to have some tools. And I think the tools need to be flexible enough for you to work in the way that works for you. Because you cannot be constrained by a tool. So, for example, when we talk about collaboration, it's really important to have a tool that enables you to collaborate with everyone in a flexible way, not in the way that everyone works. When you talk about testing, it's very important that the tool that you use to manage your tests give you the visibility that you need to share the information across the team with developers, with testers, whatever. So that's really also important. And for example, if you want to experiment or you want to have some ways to, for example, do some exploratory testing, you need a tool that helps you with that. And I believe that here Sergio you can help more than I about this, but I leave you to your opinions and say what you think about this.

Sérgio (36:39): So yeah, I was thinking about where tools are more on the testing side. And I think that, as you said, if you forget a tool, and you put there and just in place and hope that it will solve our problems, that won't happen. But, of course, we have tools or ways of getting visibility of the test automation, or that can help us implement test automation, that can help us be more efficient, that can help us not just see the results, but also track the impacts. Because whenever we think about developing software in the century, you're trying to see what is the impact of what we are delivering. And that those impacts may be related to some stories, to some help, extort some themes. And we need to be able to track that in a more or less automated way. So tools, even Xray, for example, can help you a bit on that. And also I like that we need to have some time for ourselves, for human thinking, and to also take advantage of experimentation. So for example, we have Xray Exploratory App well, it won't replace you as a human for exploring the system, but it's there to help you out to assist you while you are there. Because it can help you out on taking notes, and making the report, and taking screenshots, or recording videos, taking evidence. So you don't lose your focus while you are exploring the system. And the tool is not there to really replace you, it's there to leverage and to augment your own capabilities. So whenever we think about DevOps, we need to think of own tools that can augment our capabilities, not really replace our own capabilities. It's about augmenting.

Szilard (38:46): Yeah, very much agree on this area. And when I think myself, “okay, what are the best tools?” Of course, you cannot name one, not like people say, "Of course we are DevOps, we use Jira" Not really. But tools that are actually supporting you to be effective, support the flow, and support transparency, and doing all this in an efficient way. Those are the best tools, right? So really, those that, as Sergio you said, serving the person, serving the team, serving the teams to be high performing. And even it might be an order tool, if it's fit for their purpose. Why not to go with that? Maybe the new tool is so complicated that on short term, it's not the best way to go for, or maybe the tool is a piece of paper on the wall because that's the simplest to use and gives you transparency enough.

Szilard (39:45): So it's really hard to say and of course transformation doesn't start by integrating a new tool but on the other hand, you need to select your tools best and you need to understand from what you can select and what are the possibilities and areas to choose from. On expiratory testing just to comment, nowadays instead of the exploratory testing, we, even more, talk about in-person testing, to highlight the fact that it's done by the human because the human aspect is most important in expiratory testing. When you explore how the system behaves, it really does design on the fly, learning about the system on the fly and deciding what is the next step. Next test, you run on the fly based on the learning and knowledge that you gathered. And yes, it's really in-person. And I like that because, again, it doesn't matter what tool is there, but Sergio, you said, it needs to augment the person to be more efficient.

Lauri (40:52): There is this tool of conversation, which is basically what is the best tool for a purpose? But then you could also say there is this meta tool conversation, which is, what would you be looking from a tool? There's, Szilard, you said, effectiveness, flow, and transparency, support efficient way of working, something like that. If you were to look at tools in a meta level, and ask, "What are the characteristics of the tool that you would be looking at?" Is there a way to say, "I'm not going to give you a recommendation of a tool, but I'm going to give you a recommendation of the characteristics that you should be looking for in the tool?

Szilard (41:31): I think I'm not the technology guy. But what we are always looking for is how well you can integrate that to your current ecosystem. How well-standardized APIs is it using? How easy it is to learn it? We discussed about the efficiency, and maybe how easy it is to change it later on? Are you going to marry with that tool for the rest of your life? Or is it just helping you now and improving your knowledge, but then you can step away? So I would say these are the meta-level characteristics that you need to think about. And then anything in that one is probably the best for you.

Sérgio (42:15): I would say that we want something light, but with taste. I mean no 787 tools, we need tools that we can easily try out or even replace, or evolve, whenever needed. We need tools that are not a burden, that are light enough, but still, that can provide us real value. So I think it's about flexibility, freedom because we don't know the things we're going to need more. So we need tools that are flexible enough. It's about also enabling CLI tools, where as much as possible to have CLI tools, I would say and it's a win because Devs and Ops really love CLI tools. Why? Because they are easy to call, they are easy to integrate with, they also facilitate automation. Integration and visibility are crucial. We know that teams nowadays need to go up to several places, I think around like five places according to an Atlassian study, to understand the status of their projects. And this can really become overwhelming if you're to shame girls dramatically. So we need to have different levels of visibility and tools that can facilitate on that. And we need to think wisely on tools and we need to understand our challenges. And search for tools that will help us overcome those challenges. We don't get a tool just because someone says to get a tool, or else we will lose agility.

Lauri (43:56): Yes. Somebody has said that it's a good sign of a tool when it's been misused. Like somebody takes the tools and applies it to the purpose it wasn't originally used and to simulate that comes to my mind when you go with other family members and you open their drawer where they keep their forks and knives and spoons, sometimes you see that for instance the knives are twisted at the end. And the reason is that it's the most accessible screwdriver you can find in a household. So you always go to your kitchen and look at okay which of these table knives go in as the best possible replacement for the screwdriver I don't have available right at this point. And then if you run out of spoons, all I have to go is walk to the sandbox in the backyard and start shoveling sand because that's what the kids have done. They have gone to your kitchen and they have taken one of your silverware and go back to the sandbox to have some playtime. The other perspective, and Sergio, you mentioned that magic word and that is the automation because when you take automation far enough, then these tools cease to be, you don't see them anymore. It's almost like a great technology that you don't see it exists. All you see is the user experience or the customer experience, the technology becomes completely invisible. And I think automation does some of that for the tools as well. So what's your take on this aspect on automation?

Sérgio (45:27): Well, some may think that automation is like a silver bullet for the every problem that we have, but I don't think that happens. But the automation really can help us by helping us implementing this shorter feedback loops, for example, and pull requests, or commits by having lots systematic processes, because it's really important to have this systematic process implemented. For example, during the packaging in the deploying part of the pipeline. And also automation help us to have more time for the thinking for deep expiratory testing where our brains can use it to tackle then the unknown because for tackling the known parts, we can use our test automation scripts. Because that's basically what we use them for. And there are some things that we can only do with automation. For example, if we need to measure performance, for example, or low testing, we need to have tools. We cannot do that without tools. And if we need to quickly check our performance regression testing, we need to have tools in place to have test automation in place in order to have this short feedback loops.

Lauri (46:53): What's nice about the way you approach that it's something that is probably not only specific to test automation or quality assurance, but you can take some of those themes and apply them or more.

Sérgio (47:05): Yeah, I think many of us think on, well, maybe because we may have come from the testing background. With automation, we just focus on test automation. But there is much more than that. There's automation around the build process. There's automation on getting things to production, there's automation that can help us switch environments quite easily. That can also help us track, let's say, what's the progress? What's the status on those environments? And I was mentioning earlier, for example, Xray, where you can use the environments concept to track how our given environment is in terms of coverage, given the latest results that you obtained for that environment.

Sérgio (47:50): So there are many aspects where for sure, automation is really a benefit. Because if it's systematic things we can remove human errors there and when some time because we want to use our hands and our thoughts where we can actually provide more value. So I would say that automation is one part whenever we think about the aspect of DevOps, for sure needs one crucial part, but it's actually much more than that. Unfortunately, some teams think, "okay, we have a DevOps role or DevOps engineer." What that folk does is basically implement some scripts- automation scripts, and some magic. Actually, we don't need magic in software, we need people, everyone within the team to be able to use magic on a daily basis. Because we don't want to why not? We want to make those things natural. So automation should be another skill that the teams should should be able to take advantage of. And it's not that the conversation around DevOps should be around automation, but for sure, automation plays an important part because we want to be more efficient.

Sérgio (49:19): And we cannot be efficient if we don't automate some tasks. Let's say, I mentioned that Szilard, probably you and I know Sofia, for sure also did working the best for some organizations that want a bunch of reports, like reports with A, B, C, and D to send to the manager, C, D, and F. And another report. Well, probably no one will read those reports, but someone would need to assemble them and lose tons of time assembling something that probably would not be even reread. So the best would be to remove all those reports altogether. But in between, because well, the managers are still there, or else, you don't have like a job, eventually, you have to deal with it. In the meantime, eventually, you can automate those. So you can focus your attention elsewhere. And let's hope that at least you can make those reports a bit more tailored. You can meanwhile, try to figure out the needs of the manager of the person that will read those reports, and not have, let's say, a bunch of stuff that nobody cares really about. But if you need to do that, why not automate it, because you don't want to be doing that by hand. And I remember that we are talking about tools. And there's one tool, which is like a silver bullet actually for everything. And you know what that tool is, which tool am I talking to?

Lauri (50:57): It's a born-again shell.

Sérgio (51:00): Wow, I really like Bash a lot. But I was thinking more long Excel. Everyone does everything with Excel. Excel is like a proxy for everything. And even in some Agile teams, they use Excel for doing stuff. Well, it can be something that can help us in having busy visible gifts, like a shared sheet or something. It can be like a meantime solution for having visibility of some stuff. But probably there are better ways in order to implement those than trying to use a tool that for sure is not the best fit for what you need. So whenever think about automation, it's not about automating things blindly. It's about thinking wisely. Why are we going to automate? What are the benefits of it? Is anyone going to take advantage of this? Can this be replaced somewhere in time by something more efficient,

Sofia (52:10): I just want to take what Sergio said at the end about thinking on what you want to automate because from my experience, as you mentioned, the reports, for example, what happens frequently is that people just want to automate a process, for example, and want to do a lot of work making this truly automated. And then you just have to ask this question. Why? So that's the most important question that you have to make when you start to look at something that you want to automate because maybe in the end, you don't even need that process, or you don't need that report or whatever, you are taking a lot of effort automating it, and then you need to maintain it, because it's going to need some maintenance in the end. And you should ask first the question, why are we doing this? Why do we need this and I think that some sometimes this is something that it's often forgotten when you start automating.

Sérgio (53:18): Just before passing the ball to Szilard, just one brief aspect is, and probably will touch this topic of observability. But we need to see how the team works to understand why what the team is doing, but most of all, why they are doing it.

Szilard (53:35): Absolutely the why is a very, very important part here. And then I would even say that maybe we shouldn't talk about automation, we might talk about codification, how can I codify this problem? How can I make it in a way easy to modify, easy to repeat, easy to do it again, but when I need to change it easy to change and do it in a different way? So I think the whole codification, utilization, everything like that, making things more codified, infrastructure as a code and all those aspects in DevOps, making us able to automate these things. But still, it's helping us to be more effective and more efficient, again, to do repetitive tasks instead of us. But many times I'll be talking, let me come back again, for test automation. Szilard (54:27): There is always a question what is better to automate the big complex scenarios and we spent a lot of time on automating them, or I just want to automate those small things that speed me up so that I can do the complex thing as a human. I can creatively create tons of complex scenarios with augmented, again, I'm using that word. I like that word from Sergio, augmented with automation or tools. So sometimes we overthink our automation as doing it absolutely automatic on its own and everything and replacing humans and whatever. But I think the line is not as far as that we need to find the proper solutions that help us to be more efficient and more effective. But keeping in mind that maybe we don't need to automate everything.

Lauri (52:21): We have been talking so much about what DevOps is not. But maybe we still owe responsible audience what DevOps is about. So let's do a round-table starting with Sofia Sergio, and Szilard. What is DevOps about and keep it short, so audience can remember your definition.

Sofia (55:41): Okay. So I think that DevOps is about a culture with continuous improvement in his heart, and where visibility, constant, and fast feedback is taken in consideration to improve the system, or the software, or the service, or whatever you are providing, where experimentation and learning is part of the DNA of the teams. So you have space to learn and to experiment, and therefore to improve and deliver more value. I think it's about that.

Sérgio (56:18): So I think first, DevOps is about efficiency, but it's also about increasing our own responsibility from ideation up to production. But we are not just increasing efficiency and responsibility, we are also increasing our openness to failure. DevOps teams embrace failure, because failure exists, and there is always room for improvement. DevOps teams aren't scared about fears so much because they implement all the necessary mechanisms making use of automation that we talked about, or some other tools to be able to access them on taking decisions.

Sérgio (57:00): DevOps is about having the necessary skills and tools within the team to enable fast decisions. That in the end is also about having better knowledge, or the necessary means to increase our knowledge. That's why teams not only automate parts of the process, but also they put in place the necessary monitoring and observability kind of capabilities. And they also address operability. Because they need to be able to operate and react fast the way they want. Operability is important to these teams and relates to the Ops part of it. But we also need to think about their ability to develop well, with short feedback loops and fast, they’re alert to the value we are seeking. Their ability may require the ability to quickly spin off environment to have real time insights about what's happening in different infrastructure. Well, but also from a business perspective. Did I invent this new word drivability? If so, I should probably write a blog about it. If not, well, credits go to the community.

Sérgio (58:13): All of these are not new ideas. All of these are nice, the teams have and should be addressed. Another concept that came to mind during a previous chat that I had with Szilard is about 360 degrees and multi-layer of observability. What is that? It's actually much more than monitoring. It allows us to make questions at any moment, questions that we have never thought about. And it's meant not just for developers, it's also meant for the other team members because I want to be able to drill down from a business goal and to see how that composes into the features and the usage of those features. But I also want to do with the other way around to know of certain problems where support tickets are connected back to the business goals. So it's all about connecting business with Development, and Operations, and Support. And I will now pass the ball to Szilard, maybe you want also to add a bit on top of this.

Szilard (59:23): Yes, we discussed last time a lot about how important this is, of course, to have monitoring and telemetry in place. So we see that our systems are up and running when we talk about quality right. So they are still in production, maybe also saying shift right. So working well in there, but also we need to understand where we are able to solve the problem. So if we go back to the design thinking, did we solve the problem that we identified as a problem worth to be solved, and did we solve the problem in the right way. So for example, if we had the functionality or a feature, and we had the benefit hypothesis, why do we do this? Are we able to learn if we got that benefit? Do we have this observability good enough that we really see how the end user behaves? And is the change in the end user really driving towards the goal that we wanted? So do we have the business people also the product management, product owners seeing what they wanted to see? Szilard (1:01:05): And I think that's extremely important to say again, building transparency in the system and having the full visibility, not only in the development phase and activities, but on the business level as well how the system or solution or services being used, and how to evolve it. And that would come back to my definition of what is DevOps, and I believe DevOps is about amplifying learning to become more Agile, and to be able to react to the needs of the customers, sometimes even before they know they have a need. So that will be my view on that.

Lauri (1:01:26): Thank you. It would be a shame if we didn't have a podcast episode about your 360 observability concept. So consider this an invitation as soon as you have the synopsis and questions handy.

Lauri (1:01:40):