Considerations

Automated integration pipelines mean that we can perform integration more frequently and at lower cost. Increasing the frequency of integration reduces the integration batch size (roughly the number of changes that must be integrated over). Small batch sizes means there is less opportunity for incompatibilities to arise in the build. Where incompatibilities do arise, they are typically small in scope and therefore easier to resolve. The number of integrations is much higher, but the time take for each integration is much lower. The result is a positive contribution to the efficiency of teams.

Automated integration pipelines include automated regression testing that effectively prevents defects created by a new build from escaping into production. Using practices such as Behaviour Driven Development (BDD) or Test Driven Development (TDD), the scope of automated testing is routinely expanded as new code is added or existing code is refactored. This helps to ensure that defects associated with new or refactored code do not escape into production. The result is a substantial increase in the quality of the product, a reduction in (typically elimination of ) failed changes, improved reliability of code in production and a fall in the number and severity of operational defects.

Levels

Green

Automated Integration Pipelines

Integration is fully automated with regression testing embedded within the integration pipeline.

Ways of working include practices to extend automated tests to evaluate new or refactored code as these changes are integrated for the first time (Behaviour Driven Development, BDD or Test Driven Development, TDD).

Ways of working include practices to evaluate and resolve a failed integration so that integration is not delayed and integration batch sizes are kept small.

Amber

Partially Automated Integration Pipelines

Some integration is automated, but manual activity is also required. Batch sizes increase as a result and integration becomes more time consuming.

Automated testing may not be sufficient to fully prevent the escape of defects into higher environments, including the production environment. Testing is not routinely extended to cover new or refactored code.

Failed integrations are not routinely analysed so that errors recur, integrations are delayed and integration batch sizes grow over time.

Red

Manual Integration

There is little or no evidence of automated integration. Integration is infrequent and irregular, so that integration batch sizes are large.

The resolution of incompatibilities is extensive and time consuming.

There is little or no automated testing to prevent defects from escaping into higher environments. Defects frequently escape and result in rollback work, emergency fixes or other unplanned work.