TL/DR

Assessments of team health are well known. They help the team develop insights into their culture and rigour and discipline. Where these are lacking, the team can decide on priorities and actions for improvement. The health check is a guided form of Inspect and Adapt.

The growing popularity of Product Thinking and the use of DevOps to support long-term product development and service delivery leaves us facing two challenges. First, the range of Rigour and Discipline that we need to examine is substantially increased. Second, product development often involves multiple teams supporting the same product.

These are the motivations behind the development and use of the Product Health Check.

Product Development

Product Development is distinct from project based development in several ways. Notably, products have an extended lifetime during which the organisation actively maintains the intellectual property encapsulated in the product. In contrast, projects have a defined, finite lifetime. After the end of the project the intellectual property is used in production, but maintained only sporadically and with uncertain quality as defects in production are identified.

Teams who focus on Product Development must be stable enough to sustain the extended product lifetime. They must encompass all of the capabilities required to support the product across its entire lifecycle. For these teams, the health of the team is even more important than for other “agile” teams - simply because of their extended lifetime.

The context in which these teams operate is also more complex and more important. The product context covers a greater range of technical capabilities, teams, involved stakeholders, technologies and lifecycle statuses. This is where the Product Health Check becomes relevant, it has been designed to examine the full complexity of product development.

Key Activities

A Product is the focus of simultaneous activities that we must take into account if we are to meaningfully assess the health of the product. The work encompasses the development of new features and the maintenance of existing features and the transition of these features into production. This scope is analogous to that undertaken by more traditional project teams. When we focus on product, our work also encompasses the continuing operation of the services provided by the product and the long term improvement of the product, its features and the quality of service provided. We categorise these activities as:

• Creation of the product and its features

• Transition of the new or modified features into production

• Operation of the features to deliver services to our customers

• Improvement of product quality, service quality and ways of working

For each activity we define several indicators that will demonstrate whether the product is in good health or not.

Create

Vision and Purpose - is there a collaboratively sustained, shared vision and purpose for the product?

Roadmap Quality - is there a collaboratively sustained, shared roadmap for the product?

Backlog Quality - is there a sustained backlog for the product that is aligned to the roadmap?

Intent to Enable - is the product architecture centred on services that enable customers to self-serve?

Engineering Capability - is the engineering capability sufficient in terms of scale, range and depth?

Experimentation - is experimentation at the heart of the way that the product is developed?

Transition

Delivery of Priorities - are features delivered following priorities established collaboratively with the customers?

Integration Maturity - is the product quality assured through continuous integration with high levels of automated testing?

Change Maturity - is the product continuously deployed so that incremental value is delivered rapidly to the customer?

Operate

Service Maturity - are there well defined, effective processes for receiving and resolving defects and problems?

Service Effectiveness - are needs of feature delivery and service delivery well-balanced for the product and its services?

Instrumentation By Design - solutions are instrumented from the beginning to enable high levels of automated incident handling

Value Realisation - is the value delivered by the product’s services measured and taken into account when work priorities are established?

Metrics Maturity - is appropriate data routinely gathered, analysed and used to influence decisions?

Improve

Seek and Use Feedback - is feedback sought and used routinely to improve both work and product performance?

Improvement Led By Data - is data routinely used to improve both work and product performance?

Understanding Technical Debt - is there a clear understanding of the scale and complexity of technical debt and the value of repaying it?

Repaying Technical Debt - is the repayment of technical debt routinely included in the plans for delivery of the product and its services.

Naming the indicators is all very fine, but we need to go further to provide meaningful guidance so that teams can work together to assess the health of their product.

Defining Indicators

To provide a basis of comparison, we must describe the indicators so that teams understand the Values and Behaviours they are being asked to evaluate. Each indicator is described along with the considerations that arise for the practices of the teams.

Simply describing the indicator is insufficient. We also need to define the levels that describe how well the teams support the indicator. Different levels are described to assist the teams in objectively assessing their product against the indicator.

Click on the thumbnail image to view the indicator in detail.

Applying Indicators

Taken together, the indicators provide an objective, comprehensive and consistent framework for product teams to assess the health of their product. We encourage all the teams engaged in the development and operation of the product to collaborate in the analysis and subsequent action planning. This means that improvements are identified, prioritised and agreed collaboratively.

We strongly advocate for the application of the framework to entire products, irrespective of the number of teams that are responsible for the product. This helps to avoid local optimisation where improvements are made only to a part of the work with potentially negative impacts on other areas of work. It also reduces the temptation to use the framework for direct comparisons between teams. The analysis, its results and its use for improvement are entirely the property of the team. The interest of Leadership should be limited to encouraging teams to use health checks for objective evaluation and to use the results of evaluation for improvement.

Product teams may choose to share the results of their evaluations and their subsequent improvements with other products. This sharing of information can improve the speed of analysis and improve the choice of improvement activities. But the context of each product is very different, thus it is an error to believe that an improvement by for one product can be applied successfully (or, perhaps, at all) in the context of another product.

Prioritising Improvement

When the teams prioritise the indicators they will improve, they focus initially on two factors. Importance - how critical is the indicator to the achievement of good outcomes for the product? Keeping things simple, the teams achieve consensus on whether each indicator is important or less important. Urgency - do the teams need to act quickly to improve or sustain good outcomes for the product? Again, teams seek consensus on the urgency of each indicator. Further analysis focuses only on indicators that are both Important and Urgent.

Taking only the important and urgent improvements into account, teams identify potential improvements and use these to identify how they can achieve the biggest impact for the least effort and cost. We have the basis for prioritisation. But prioritisation is as much art as it is algorithm. We also need to take into account factors such as feasibility and executability.

Feasibility looks at the potential to make the change successfully. For example, Is the organisation even able to contemplate the change being proposed? Executability looks at our ability to implement the change - this may be impacted by the capacity of the teams to divide their effort between building their product and implementing change.

Use the content link, below to access the Product Health Check pages.