Measuring success beyond numbers in platform teams

Part 6

"A platform team’s value can be measured by the value of the services they provide to product teams." - Skelton, Matthew; Pais, Manuel, Team Topologies

The challenge of traditional metrics

When gauging a team’s success, we've often seen the introduction of DevOps Research and Assessment (DORA) metrics.

For the uninitiated, DORA metrics, which include measures like deployment frequency and time to restore service, act as an industry standard for assessing DevOps performance.

Pushing such metrics from upper management down to teams can feel like a cold shower.

Why? Because some teams fear "naked metrics." There's a certain vulnerability in being transparently measured, especially if there's a chance of an unfavorable comparison. Let's face it, no team enjoys being compared to another, especially when it's like comparing apples with oranges.

Different departments have their own challenges, e.g., DevOps teams will have separate focuses to project managers or platform engineering teams. Each possesses distinct strengths. So, making a straightforward metric-based comparison can be, at best, misleading and, at worst, demoralizing.

Owning metrics in software delivery

Now, if the traditional top-down approach of implementing metrics isn't ideal, what’s the alternative? The key is for platform teams to own their measurements.

• Understand the metrics: Before anything, the team should grasp why certain metrics are important. Take DORA metrics, for instance, explained brilliantly in this Google Cloud piece. By understanding the rationale behind metrics, teams can authentically embrace them.
• Incorporate SPACE Metrics: Introduced By Nicole Forsgren, SPACE stands for satisfaction, performance, activity, community, and evolution. These metrics encourage a broader view of success, incorporating technical and human factors.
• Learn from thought leaders: Thought leaders like Abi Noda provide insight into how metrics can serve as genuine performance indicators without becoming punitive tools. The wisdom lies in viewing measurements as guideposts rather than stop signs.

Trends over absolute value

Is it good if your platform team improves its deployment frequency by 10% in a month? It might seem so at face value, but it's hard to say without context.

That's why trends are paramount. Rather than focusing on the snapshot of a specific metric, track its movement over time. It's the trajectory that provides insights, highlights growth areas, and showcases genuine progress.

Example measures for platform teams

Here is a list of metrics to consider when gauging a platform team's effectiveness and identifying areas for improvement:

1. Platform adoption rate:
Description: Measures how many product teams or services actively use the platform against those that don't.
Calculation: (Number of product teams using the platform/total number of product teams) x 100.
Why it matters: A higher adoption rate can indicate the platform's usability, relevance, and effectiveness. A lower rate might hint at potential challenges or gaps in the platform's offering.

2. Mean time to onboard (MTTO):
Description: Quantifies the average time it takes for a product team to onboard and start actively using the platform.
Calculation: Sum of all onboarding times for product teams/number of product teams onboarded.
Why it matters: A shorter MTTO suggests that the platform is intuitive and well-documented. A longer MTTO might signal challenges in platform understanding or indicate missing support materials.

3. Platform service uptime:
Description: Measures the reliability of services provided by the platform.
Calculation: (Total time, downtime)/total time.
Why it matters: High uptime percentages indicate a robust and reliable platform, crucial for product teams who depend on it for operations.

4. Feedback loop duration:
Description: Tracks the average time from when users provide feedback to when the platform team addresses it.
Calculation: Sum of all feedback response times/number of feedback instances addressed.
Why it matters: A shorter feedback loop can signal an agile and responsive platform team, indicating their dedication to continuous improvement and user satisfaction.

Starting with these metrics can give platform teams initial insight into their performance and areas that might require attention. Remember, the key is not just to measure but to act on these insights to foster growth and improvement.

Top tip: Backstage offers metrics and KPIs related to the effects of launching a developer portal, which could provide valuable insights.

Metrics aren’t just numbers on a dashboard

Metrics are stories of growth, challenges, triumphs, and learning. For platform teams, they serve as a compass. But remember, while the compass gives direction, the team writes the narrative.

Embrace metrics as a tool to guide, enlighten, and evolve, ensuring the path to success is measurable and meaningful.

It's now time for the final blog post in our platform engineering series—communicating achievements and challenges.