The best metric to start with

“I know there are a lot of metrics out there, but I’d like to start with something. What’s your experience? Which metrics do you use to measure progress and effectiveness?” – from time to time we hear this kind of question from our Clients, especially if their business-related experience exceeds the technical one. As well-trained Agalistas (where did I find this term? Too much Taleb, I suppose), we immediately recall Story Points/Ideal Hours and Team’s Velocity as the natural choice. But the real question is if these are the best ones? It takes a lot of “ifs” to answer it – from team maturity, through customer maturity to project/product phase, and so on. But to simplify complicated by rephrasing the question (which is, by the way, a part of Agile Manifesto – “Simplicity”) let’s assume that we would like to have just one metric, which would help us navigate through the uneasy waters of the project. What would it be? My favourite choice is Cycle Time and below I would clarify why.

At first some theory

To start let’s make clear what Cycle Time means, using its definition. According to the Cambridge Dictionary, Cycle Time is “the time between the beginning and the end of the process of making a product”. In this form, it’s highly Client-centric, but in the further part, we will adjust it for the viewpoint of project management.

Although this may seem obvious at first glance, it includes the next terms, that should be defined using subsequent definitions. However, we don’t want to fall into a recursive loop of definitions, so for the sake of this text let’s just assume that the “product” – apart from the obvious, finished ones, e.g. ready to deliver car – can be a part of something bigger. That within our domain would be a single feature, a component of the software in progress. What I want to also emphasize before moving further is that this metric reflects the time from kickstarting the work till it’s done. That’s why this definition may sometimes be linked with another metric – Lead Time. The best way to separate this two would be with a simple graphic like the one below (if you want to assume that it is not a Kanban board, you have the right to do so – it’s a demonstrative board :))

The business value of Cycle Time

As seen of the graphic Cycle Time starts in a moment when the team commits to start doing some specified work. That makes it a crucial metric for the business. Even if Clients ask, sometimes simply courteously, about the way the work will be done, from the strick business perspective it doesn’t matter who will do exactly what, and how each stage of the process would look like. What really matters for the business is when the new feature would be available and what are the potential costs of delay. And the answer to this question can be found using, yep, you guessed right, the Cycle Time. It displays when the work the team commits doing will be finished (friendly reminder – we’re still considering a single feature; batches are a different story) and how long would it take. But since we have different Cycle Time for different elements of the project that the teams are working over (depending on their complexity), now you may wonder which value should you share with the Client?

Read also: Essential KPIs in software development

Within knowledge-based work, and software development fits this type without a doubt, it could be the mean value. However, that comes with some flaws, especially if we’re about to use it for forecasting. Additionally, if the mean values are applied to set deadlines, that often creates lots of misunderstandings on different communication channels. All of that makes using a graph created with Cycle Time values for all realized parts of the product/project a better choice.

Cycle Time Scatterplot

Let’s briefly describe the graph – the horizontal line represent time while the vertical line is simply Cycle Time, where the highest value is the time measured for item/requirement/story that maintained longest in the production process. Dashed lines show percentiles, giving immediate information about where a given value is in relation to another (yes, it is simply a definition of percentiles ;)). When translated into business language, it simply means the probability with which a given functionality will be completed in a given time. To illustrate it with an example – if the vendor looks at the bottom portion of the graph below the 50% line and on this basis promises something to the customer, there is a high risk of having a serious conversation with the customer after the declared time. Why? Promising that you will deliver particular feature in no more than 7 working days is risky, because the probability that you will succeed is less than 50%. This kind of visualization helps to facilitate conversation in terms of probabilistic thinking about the deadlines instead of deterministic based on averages (by the way, Velocity is an average). What I observed using Cycle Time with scatterplots is a significant reduction in wishful thinking on stakeholders’ side. It can be clearly seen what the probability level is related to on-time delivery, inherent risk, and how these values can change using different scenarios.

Therefore, instead of saying that something will be finished in seven days, we can say that there is an 85% chance it will be finished in 11 days or less – depending on what risks we are able to accept and the costs related to delay (although the choice of values is an arbitrary example based on the above graph, this percentile value is quite popular).

Additionally, based on the same graph, outliers can be easily spotted. Even though they can be removed for the sake of statistical analysis, they are very good reference points for retrospectives. But we will focus on retrospectives in another post, so if that seems interesting remember to come back here every once in a while, 😉

One metric to rule them all

Moving further, Cycle Time let management focus on improving the process, not the people, primarily. The question is, what can we do to shorten Cycle Time (of course if that’s the business goal, another example might be improving quality while maintaining the same Cycle Time), instead of pushing all the people to work at full capacity. These are two different things and rather than explaining it with another block of text just look at the picture below – which cars have a higher chance to leave the highway earlier, on the left or right side of the road? ?

Here’s the free spot, let’s put something there we’ll have 100% capacity, hooray. But wait a minute, isn’t the goal to move?

Analysis, production, deployment (even within teams working in Scrum it happens sometimes that deployment is being covered by people from different teams), people working in the following stages often operate in silos and throw the half-product over the proverbial fence, based on a simple principle “we’re done with our part”. Even if there are any steps taken to improve the process, there is often no single global metric that is monitored in connection with these. When carrying out any improvements, changes in the process, competence changes in teams, or structural changes, regardless of where they occur, it is good practice to observe the whole process not only it’s a single part. And again, Cycle Time works very well in this role.

Here comes a very interesting, non-intuitive issue. When we look at the visualization of the process (especially one that does not take into account queues – no Waiting Time visualization), we will undoubtedly see how fragments of the product go through its individual stages and immediately get the impression that a lot is happening, and the work is going well. Well, refrain yourself from a celebration for a moment since it may turn out that we’re completely wrong and in order to realize this, we should use a simple equation that shows what is the percentage of active work in relation to the whole Cycle Time called the Flow Efficiency.

Total Cycle Time – an aggregated time measured for all elements that have passed through production for a given period. All Wait Time – an aggregated time of blockers, waiting for others in the team to finish their part, etc.

Quite quickly it turns out that the time we don’t work is more important than the time we work. Reducing Waiting Time is one of the first and best improvements that can be made to improve the whole process and reduce Cycle Time.

I especially recommend this little bit of algebra to Project Managers in large projects, where there are at least a few stages of the process that happen outside their direct control, i.e., where additional work is being done on the suppliers’ or the customer’s side.

Where to apply Cycle Time?

The Cycle Time can be used on many granularity levels: Epic, User Story, Task, and is agnostic to management framework or methodology. Whenever there is one artifact transformed into another there must be some process in a place. Input – start time, output – stop time.

To sum up Cycle Time:

• The metric is utterly understandable for both parties (provider, customer) discussing the new functionality and its delivery date. Apart from the main question “how much will it cost”, it is as important when it will be done.
• If the right visualization (scatterplot) is implemented, decision-makers can quickly notice that making decisions based on average values is not the best strategy for determining deadlines.
• Due to the fact that with the information about the time needed to craft a particular part we also have the information about the probability, business departments can estimate the expected value for diverse scenarios with different risk levels.
• It is easy to measure it between any two points in the process and thus optimize those places that are responsible for the highest percentage parts of Cycle Time
• By its very nature, it is a global metric, which encourages thinking in terms of optimizing the entire process and breaking down silos between teams, departments, etc.

To gain all these benefits we just need three things:

• Good, real work reflecting process visualization implemented in some kind of software or on a wall or white board.
• Whenever we work on something for the Client it should go “through the production process”, and
• A little bit of discipline to update items status according to real status in production.