Surprising truths about DevOps: trends, tools and fads

DevOps is an ideology about what is efficient software development, what company culture supports those efficiencies, and what the ultimate end goal of your software development is.

More broadly, though, DevOps can also be seen as an ideology on how to tackle emerging threats. These threats include role of nation states in an evermore globalized world, major financial upheavals that statistically are bound to happen, or the ecological time bomb that all of us are sitting on top of. If you can’t create software to respond to threats quickly, that’s going to hamper your response. Let’s take a look at the trends, tools, and fads of DevOps.

The trends

Visibility, Cloud, and Virtualization: The starting point

• Visibility and transparency: Tools & platforms enabling data collection (Elastic stack), visualization of data across different disciplines (Grafana), creating multifaceted, data-driven metrics (data science, DataOps).
• Cloud adoption will only increase in the future. The data center is arguably dead. IoT especially will be the next battlefield among Cloud providers.
• Virtualization & automatic environment management: Docker expands and soon, it can be taken seriously as a platform for Microsoft technologies. We’ll also hopefully see virtualization tools able to mimic entire third party systems, easing development of interconnected applications. For example, if your application needs to authenticate users from a national authentication provider like Tupas, you could have a pretend version of entire service provider – a mock – running on a developer’s laptop so they can develop the software in a real-life scenario.

Before virtualization, developers and testers actually needed a shared server that is actually connected to the third party to complete their work.

What about Gartner’s trends?

Another authority on DevOps trends that regularly publishes DevOps trends is Gartner, an American research and advisory company.

In 2019, most of the Gartner trends listed revolve around the same larger domain of infrastructure agility. It shouldn’t be the bottleneck when a new product should be launched in a matter of days. Losing this bottleneck leads to being able to handle potentially global, sudden user traffic when services go viral, without compromising on the quality (including, especially, the security) of the product.

Going back a little further, in Gartner’s 2017 Enterprise DevOps survey, team culture was one of the top attributes listed that determines an organization’s ability to scale DevOps.

Another thing to note about Gartner trends is that few of them are new – these have been on-going, multi-year trends that will be relevant for years to come. It shows that, generally speaking, most companies are in the beginning of their DevOps transformation “journeys”. That means the predominant question around these trends is “how to start”. Few companies are so far along that they can ask “what else is there” or “how do we go further”.

Conclusion: a sound foundation in DevOps ‘basics’ will leave you in good stead here.

But what is new in DevOps? Like, actually new?

As always with progress, things keep trucking forward at a speed that’s hard to notice if you’re in the midst of it all.

In many areas, tried-and-true DevOps practices are entering new business areas like heavy industries that previously have not known the possibilities of creating software in this way. Automotive, chemical, and electrical industries especially are becoming increasingly software-based, meaning the age of customized hardware is giving away for generalized computing hardware with the software in it providing the solution. This means there's a need for efficient software development, which in turn is what DevOps is all about.

Microsoft Azure is a Cloud provider extending its offering to IoT development. Modern IoT is almost always network-enabled so this is a natural progression – we’re amazed it hasn’t happened already.

Yet another trend that has previously been solely in Cloud, but is now maturing as a general architectural solution is Function as a Service (FaaS). In it, an infrastructure is built which is used for elastically support batch processing needs. When processing is needed infrequently – for example processing user-uploaded images to thumbnails – computing resources are automatically provisioned from nothing just for this one process and decommissioned automatically once it has been completed. Amazon Lambda is an example of this kind of service in the Cloud, but recently we have seen software solutions that help you set up this kind of elasticity to your own ICT infrastructure as well.

In DevOps, forming the right kind of culture within an organization is the key factors to being able to leverage new technologies and practices. Inner source is one such effort that is becoming more and more popular. In it, open source best practices are leveraged inside a company, meaning teams and projects can see each other’s version control, work management as well as other tools, thus give the possibility of developing a culture of sharing and innovation.

Another DevOps concept that aims to improve culture is Design System. To describe Design Systems very briefly, UI/UX design is integrated to all software development across the organization by designing commonly re-usable components for any software product to use, thus ensuring shared expression of the organization’s brand. This shifts DevOps-like practices even further left.

What is the one DevOps tool you should invest in?

DevOps is finally starting to see a host of mature tools that are battle tested. When it comes to DevOps, tools is not nearly everything. Even with all the right tools, your company culture may be preventing you from fulfilling the potential of DevOps.

With that caveat in mind, if we were to recommend one tool I’d say Grafana.

There are a lot of steps that get you from an idea to a finished product that is being iterated to become an ever-better product for your customers.

There are inefficiencies you are unaware of and they can be at any point in this process. In order for you to make the entire end-to-end process as efficient as possible, leadership needs to have real-time visibility over the process to spot bottlenecks. Grafana is a freely customizable tool to visualize whatever metrics you want to see.

As more and more companies mature to truly data-driven decision-making, tools that enable you to envision and build your own metrics will become more used in daily decisions. Exactly how C-level management is going to give up those rose-colored shades Puppet mentioned in last year’s State of DevOps report for a genuine HD view of how the business is actually doing needs to be decided on a case by case basis.

What tools or techniques may be a passing fad?

Serverless development by and large seems like it only fits certain kinds of problems. There’s a good chance you’ve heard of serverless: it’s very much been a buzzword in the industry.

But it’s not actually this revolutionary new software development practice that the current serverless hype train wants to make it out to be. Many of the tried-and-tested tooling decisions one would need to deliver high-quality solutions – like version control, testing, security – do not have satisfactory solutions when it comes to the serverless world.

Serverless is not going anywhere. But, I’d argue, it’ll only stick as an architecture pattern to certain kinds of problems. For example, applications with clear, simple, batch-like background tasks, or as a way to enable rapid prototyping of a new product.

Let’s hone in on DevOps in Europe

We’ve seen that in Europe, many organisations are struggling either with setting up basic DevOps tooling or harmonizing the tooling organization-wide. As discussed previously, one needs a solid base to achieve software development skills that are not outcompeted by competitors.

For this reason, one should start with a solid foundation of tools which are integrated together. This is what will form your continuous delivery pipeline. We’re talking version control, documentation management, work management, a continuous integration/delivery server, and configuration management.

After that, taking Cloud into use is the next big tool-related challenge. Thankfully, it’s a challenge which is far from impossible to tackle.

Germany in particular has a lot of heavy industry which is quite traditional and changes (comparatively) slowly. There are huge gains to be had, as already seen elsewhere, in how DevOps practices are employed in these fields.

Tools aren’t everything, and this is what we mean by that

Taking DevOps tools into use presupposes the ability to work according to Agile and Lean philosophies.

For example, Acceptance Test-Driven Development (ATDD) is a development methodology based on communication between business customers, developers, and testers, where requirements are recorded as executable test cases. The practice assumes that one is able to manage project requirements with user stories. Therefore, just taking an ATDD tool into use does not get you very far by itself.

Another example is Cloud computing. You need to understand how to provide elastic, variable infrastructure and not just take Cloud into use for the sake of it. Otherwise, the commonly touted benefits of cost-cutting – the reason given to move infrastructure from your data warehouse to Cloud – might not be reached. In the worst case scenario, a poorly cloud-optimized application’s operation costs actually increase compared to what you had previously. When this happens organization-wide, across all applications, the results are catastrophic.

How can you keep up-to-date with latest DevOps tools and trends?

Following hip sources like Reddit, StackOverflow, or others, does not really tell you which new technologies or tools are worth your time – often even DevOps veterans can get excited about a tool that is dead in six months, never having matured.

A far better way to keep up with progress is to build non-mission-critical projects with new tooling to gain understanding of pros & cons of, but especially the maturity, of a tool.

This is why companies such as Google have initiatives where employees are expected to use 20% (for example) of their work hours for personal projects – which makes them try out new technologies as part of their day to day work.

Making software more efficiently means cutting through nonsense

Although DevOps is entering the mainstream, it really doesn’t matter how adopted DevOps currently is.

If an organization is trying to make its software development more nimbly in the face of change, as well as more efficiently, it is striving towards the same goals as DevOps.

Even if you don’t think you’re doing DevOps.

Not employing proven practices and technologies is stubborn, and not in a good way.

This article was written by Heikki Hämäläinen, Eficode's Head of DevOps, and Petteri Ahonen, the Country Manager for German-speaking Europe at Eficode. It was originally published in German in IT Finanzmagazin.