Avoiding the shift-left fallacy

DevOps is all about shifting left and shortening feedback loops. Or at least it’s about shifting a third of the way back. Kind of. 

But, there are two massive assumptions built into the phrase “shift left.” First, we assume there is feedback available for us to shift left. Second, we assume people are actually going to consume the feedback to make it valuable. 

Unfortunately, many teams fall into one or both of these traps. They live in what I call the shift left fallacy. 

Shifting nothing left

When we talk about feedback in a DevOps context, we usually think about internal and external quality and usage metrics of any given product. This could be measuring which parts of our applications are used by most of our customers, or internal quality and performance metrics such as memory usage and test failures.

Shifting left is about taking feedback and making it available to the builders. The feedback should be quick and loud to have the biggest impact at the lowest cost. If you talk about this with a software engineer, like myself, you can be certain that we will immediately launch into seemingly random permutations of cool tools to kick the ball out of the park with an awesome feedback delivery mechanism. But there’s a problem with this. What happens when this ball-kicking robot is deployed to production and we can't find a ball for it to kick? 

Because it really doesn’t matter how cool your Prometheus and Grafana is, or how your favorite SaaS metric startup promises to make your infinitely dimensioned data trivially queryable. Take a step back and try to find a single piece of useful information. Find out how many server errors your application reported during the last 24 hours. Do it manually. Repeat this manual process a few times during a month and record it down in an Excel sheet or similar. If you want to go truly feral, grab a post-it and stick your data point on your desk. Of course, technical measurements like server errors are easy to measure. For a really valuable challenge go and ask the business owner what metrics they care about. Better yet, find a real user of your application and ask them how happy they are with it.

And if you can’t find any useful feedback your focus should be on figuring out how to get some, not on some imaginary shift left process. 

Shift left into the void

The second aspect of the Shift Left Fallacy is eliciting all the valuable feedback, getting it to where it will have the biggest impact, and then completely failing to act on it. This happens because we either just ignore the feedback or we are technologically/organizationally incapable of making it useful. If we are unable or unwilling to act on the feedback it is wasted. All the effort and energy spent obtaining the feedback is wasted too. 

If we invest in tools like XRay to scan for vulnerabilities or license violations, we can get a lot of valuable feedback. But this can result in a deluge of violations, warnings and errors that leave us paralyzed and numb to the severity of our issues. It is much easier to ignore a thousand warnings than just one. 

So, in the license case, for instance, start by figuring out which licenses are valid for you before putting checks in the pipeline. Then go through your dependencies to find a single violation and take action! Are you going to stop using it? See if you can find a commercial license or alternative product? Or, worst case scenario, develop an alternative yourself? 

If you are unable to cope with one violation you should not shift left on that particular feedback loop. 

The problem is twofold. First, you will have simply wasted your efforts in enabling this particular feedback. But, the true tragedy of the shift left fallacy in this case is that you are training your teams to ignore feedback, blunting their ingenuity, autonomy, and trust in the organization. It says a lot about us that we can invest in obtaining this kind of feedback without taking action on it! 

Feedback loops are about feedback 

Feedback is about converting the output of our system into inputs for our system. If we are not doing this, feeding and changing our system either through our users or monitoring, we should not be talking about feedback loops and shifting left at all. 

So, the best place to start is with the feedback. Then we can consider how to loop it. 

Here are a few ways to start working with feedback. 

Pair or mob program

This gives you a chance to work with your feedback and information internally. Then, once you have a handle on mob programming, you can invite outsiders such as InfoSec and QA to the mobbing sessions to help you shift the feedback left. 

Bring more people to the initial design phase 

When you start significant work make sure that everyone with a relevant value stream is present. It’s not just designers and developers who participate in generating value and who set the context for development work. It can be everyone from partners, platform teams, QA and security. Designing and architecting for security, testing and operability from the get go is a huge benefit. It’s also much cheaper and a lot more fun than trying to tack it on later in the process where wide reaching changes are painful. 

Do it the stupid way 

Start by deciding on a key business metric that you'd like to act on. If it’s trivial to measure it automatically, go with that. But, if it’s even slightly challenging, stop in your tracks and dumb it down further. 

Guesstimate the metric on a regular basis and see if that provides you value. If it does, it’s probably worth the automation and shifting left. This is also a great time to experiment with choosing the right visualization methods. Visualizations can be used to emphasize or hide trends. While you are still experimenting with gathering data you’ll have a high degree of visibility on how it is trending, so take advantage of this and make sure you can visualize in an impactful way.

It is not free to shift left 

We must strive to shift-left if we want to be competitive in the digital universe that we are living in. However, just dropping shift-left as a buzzword and disregarding the technological and organizational prerequisites will be wasteful and even downright destructive.

Here are the three signs that you have stumbled into the shift left fallacy pit and need to get out:

• You have plenty of feedback, but it is not shaping your actions
• You spend more time on the tooling than on understanding the data
• You are always wishing that some feedback had been generated earlier

So, prioritize shifting left, but beware of the shift left fallacy!