KPIs help teams to focus on the most important thing to be built and provide insights to question and improve established processes. Measuring and monitoring the right metrics is the first step towards improvement.

Before we talk more in detail about KPIs, we need to keep one thing in mind: Figure out one improvement area - take one step - measure - repeat

All KPIs are only indicators of potential areas of improvement and the team needs to have conversations around the numbers to find out if there is an improvement opportunity.

This blog post provides you with a short manual and an overview of agility KPIs in the following areas:

• Increasing business value
• Producing high outcome
• Executing really quick
• Building it just good enough
• Keeping a constant pace

Business Value

There is maybe no other KPI that well known than this one - Business Value. Each item in your product backlog - either User Story or Epic - should have Business Value attached to it. A good way to define business value is by evaluating the relative benefit (how big will the benefit be from customer perspective, when this feature becomes available) and what is the penalty for the company, when this story or epic will not be implemented. You can get started with this one very quickly. Just use numbers from 1 (low) to 10(very high) for benefit and penalty. At the beginning search for a rather small reference story,epic or initiative so that you can put all other estimations in relation to this one. Just add these two values and, as a result, you'll get result you can use for business value.

benefit + penalty = BV

Estimating the penality is an important aspect. Especially technical projects will not directly impact customer value, thus benefit will be rather low for these kind of projects. But it will have an impact on other projects, which can be implemented faster in the future, or it might solve current stability issues with a service. Therefore the penalty for not implementing the desired changes will be rather high. With a standardized way to calculate business value for feature projects as well as for technical project you can easily have technical items as well as product items next to each other in your product backlog. All of the projects will have one standardized way to measure value; your calculated Business Value.

Business Value Delivered by Sprint

By having started adding business value to your backlog items you're building a foundation for the next KPI: Business Value Delivered

Delivered Business Value is actually the outcome each team wants to optimize to satisfy the customer and is reflected in each product increment at the end of each sprint.

Measure this KPI for your scrum team by just adding the business value numbers by sprint item and plot it on a chart. You can than easily see if business value is delivered in a constant pace. If the business value delivered KPI drops, it could be an indication of missing innovation? Just a guess. But you and your agile team will figure out together what is causing it, how you can fix it and ... most importantly ... if it is actually an issue.

The turnaround time of backlog items measures how fast features can be shipped and is driven by a number of facts. First of all team utilization and also by impeding processes or technical debt. Those facts can and also should be made visible.

Team Utilization - Avg. Lead Time

Measuring average lead time can give you some insights about the utilisation of the team and it will help to identify a potential overload of one of your teams. Average Lead time is measured by calculating the time delta between the delivery date and the date when the item was added to the backlog. Also, plot a chart over time and evaluate if the average lead time is growing. If so, start evaluating if an augumentation of the team might be necessary. If you have more than one team, look into the utilization of other teams. If there is a utilization missmatch you need to look into the team setup, but also into the architecture to evaluate if specific services or features can be moved to another team.

	
deliveryDate - addedToBacklogDate = avgLeadTimeInDays
	

Impediments - Avg. Cycle Time

Measuring the average cycle time for a backlog item highlights potential process issues or puts the focus on impediments. This number shows how long the team was working on a particular backlog item. Calculate this number by:

endDateOfLastTask - startDateOfFirstTask = avgCycleTimeInDays

Acting as an agile manager you need to look closely at all numbers, of course, but the average cycle time should be of particular interest. The cycle time - tackled early enough - can lead to significant process or organizational improvements if discussed with the team openly and early. Potential reasons for long Cycle Time in agile projects can be:

• missing unit tests
• messy code which needs refactoring
• changing requirements
• big user stories
• ...

The term "we build high quality software" is often used, but very seldom specified for teams. And this holds a huge danger. The danger of overoptimization, which is seldom a good thing as it frequently slows down the cycle time and also increases code complexity. Therefore introducing Service Level Indicators is a good thing as it helps the team to focus.

Service Level Indicators

When building a new service or feature, one of the acceptance criteria should include at least one SLI. Service Level Indicators give the team the right information about how good a service or feature needs to be implemented. Let's consider the following Service Level Indicators as an example:

• Response Time of API Request: <500ms (excl. latency)
• Availability: >95%

That avoids that teams over optimize, which - in turn - leads to better quality. Simply because a feature was implemented with the right focus and with a clear expectation about its performance and availability. Besides that, you talk about one aspect of quality - operational quality - in a very specific way

A very nice side effect is, that the SLIs can be turned into KPIs themselves. Neat, isn't it?

Defects in Production

Writing software and making it accessible to clients automatically implies that there are going to be defects. Avoiding defects in production by all means is something I don't recommend. Don't get me wrong. I'm not saying defects are great, let's release all of them. No. But having some defects in production could be an indicator that you are moving in the right speed. However, it is a good investiment when you take care about your defect culture by establishing a lightweight guide which describes how to classify defects by impact and severity

• Impact - How big is the negative impact on our customers
• Severity - How hard is it to mitigate this issue

Building a graph which plots the number of incidents by impact and severity will draw a good picture about the service quality (keep in mind your SLIs). If numbers are going up you need to look into them in detail, why this could be the case.

Code Coverage

This is a delicate topic. Just to set the right expecations about code coverage and how to handle it:

I expect a high level of coverage. Sometimes managers require one. There's a subtle difference. -- Brian Marick

Find the whole article here

Thus it is a good thing to expect a high code coverage, but it should not be required. So, hmm, what is the difference.

First of all, the team needs to define their code coverage goal. When this is set in stone, configure the target value in your CI/CD pipeline and hold the team responsible for having conversations when code coverage declines. There could be multiple reasons for that. Figure out what has caused the drop and have a conversation about this topic.

And before you talk about code coverage: Define a very clear strategy how you are testing and define that on a story level. Read more about a very good explanation about how to test https://kentcdodds.com/blog/write-tests

Operating at a constant pace is a desirable state each team should try to achieve. That does not mean, that you should't try to improve and deliver more. No. The main focus here is to keep a constant pace, without high volatility in your numbers. Try to be as close to your predicted numbers as possible and still try to find ares of improvement.

Started vs. Done Ratio

A started vs. done ratio is easily calculated by counting the started tickets (not stories or epics) and also the ones which got moved into the 'done' column. Ideally this number is very close to zero as too many active items tend to slow teams down.

Velocity Standard Deviation

Another method to see if there is room for improvement in the area of delivering with a constant pace is to use the Story Points burned by sprint and calculate the standard deviation of the last 3 months. This comprises 6 sprints, when you have two weeks' sprints. Dig deeper when you see that the standard deviation of the burned story points is changing and also compare the Velocity Standard Deviation with Business Value Delivered.

During your agile journey you will find ways to improve. Things you will adopt or dismiss. Measurements and metrics will help you to set the right steps and they will visualise if you are making steps forward or - hopefully not - steps backward. However, ... Keep it simple. Get started quickly and then improve when needed.

Enjoy your journey.

P.S.: One important thing: When you try to adopt one of the KPIs keep an eye on the others to avoid that other areas suffer from the optimization.