Sustain Metric Definitions
We measure stuff because we want to base our decisions on information. Consistent information can only be created from data that is itself coherent and consistent and that is analysed consistently too. To keep our data gathering, processing and analysis consistent, we ensure we have clear definitions of the data we want to use and how we will use it.
In A Nutshell
We use our metric definitions to ensure the that we collect, clean and analyse data consistently. If our information is created inconsistently, then the meaning of the information is distorted. Making decisions based on distorted data means that our decisions will be incomplete or incorrect. We will normally need to collect and analyse data over an extended period of time. To ensure consistency is sustained over our timeframe, we define our metrics and ensure that definition is maintained. Our metric definitions can be document based. Where we automate the extraction, cleaning and analysis of data, code-based definitions may be appropriate.
Metric Definition
Our metric definition should address the following:
Definition of the metric to give a specific understanding of what is being measured
Sources of raw data values that contribute to the metric
What counts as valid data values for each of the raw data sources
Calculation of derived metric values from the raw data values
Standard analyses that will be applied to the derived metric
Indications of where we should apply interpretative notes to the analysis
Example
The following example assumes a documentation led approach to the definition of the metric. Definition by example or by code is, of course, a suitable alternative. The example is deliberately simplified, for example glossing over the detail of how the work start date or word done date might be obtained from a specific toolset.
Description
Cycle time is the number of whole working days from the point where work starts on a backlog item until the backlog item is marked as done.
Sources
From each backlog item that has the status “done”, select the “work start date” and the “work done date”.
Validation
Work start date must be a valid date. Work done date must be a valid date. Work done date must be on or after work start date.
Derivation
Cycle Time is the number of working days between work start date and work end date plus 1 day.
Analyses
Partition Cycle Time by backlog item size and compute the mean for each size. Use the z-statistic to compare the means of successive sizes. Inference, if the means are not significantly different then sizing may not be working well.
Analyse Cycle Time trend over time…
Interpretation
Note where changing circumstances have had a significant impact on Cycle Time for a given story size.
“Let’s Experiment!” I heard one leader cry. “But Why?” retorts another.
Maybe we find ourselves trying to do more with less, trying to keep pace with a competitor or even keep one step ahead. In all these cases, we will need to adapt and try new things - experimenting can be a systematic way of testing our assumptions and documenting our findings.