Forge

Problem

Measuring productivity in software development teams is hard. Martin Fowler famously wrote in 2003 that it was impossible to measure [1]. This makes it hard for management to know if they can push teams, and hard for teams themselves to understand if they are operating at a steady, but not quite optimal state.

The past 25 years of software engineering have seen many developments, from the adoption of agile practices, to the emergence of devops. In particular, since 2014 the annual State of DevOps Report [2] surveys and tracks practices and outcomes in thousands of teams across hundreds of organizations, large and small in the hope of identifying patterns of highly effective and productive teams.

Through these studies and surveys, some aspects of software delivery have emerged as being highly correlated with effective teams delivering value to their user and organization. This can be measured through 4 metrics, dubbed the DORA metrics.

What are the 4 DORA Metrics?

• Deployment Frequency measures the number of times
that code is deployed into production. It’s usually
reported in deployments per day or per week.

• Change Lead Time measures the time it takes from code
being committed to that code being released into
production.

• Change Failure Rate measures how often a code change 
results in a failure in production.

• Time To Restore is the amount of time it takes a dev team
to recover from a failure in the system.

Why do tracking these matter? The 4 metrics are interconnected, and correlated with the production stable and high quality software.

For example, when teams deploy often, it typically means they have small chunk sizes for their deployments, and that they have high confidence in their deployment processes. By having small chunks, the risks should be well understood, and thus teams can respond in a targeted manner if issues are seen. These should both reduce the number of incidents and the time to restore. 

Similarly, tracking failure rate and time to restore is important. Obviously, incidents that impact users are undesirable and should be minimized, and time dealing with incidents is time not used to deliver value to users.

Solution

Forge is an Internal Developer Portal (IDP) for OGP, which tracks the 4 DORA metrics for projects at OGP. 

Forge provides information to OGP teams without expensive setups. The project is feasible because teams at OGP all use git and github as their revision control systems, and have similar automated ways to deploy their project to production. Additionally, teams track incidents in a central manner for shared learning.

Tracking DORA metrics can help drive conversations within teams. Team can make educated decisions on what they could be focusing on next, and they can monitor their progress.

For example, the metrics can be influenced by a variety of setups and practices that teams can consider adopting. Here is a non-exhaustive list below:

• Introduce automated test coverage

• Automate deployments

• Reduce chunk size

• Reduce work parallelism

• Implement feature flags

• Set up robust monitoring and alerting

• Train for incident response

• Always run in-depth, blameless incident retrospectives

Here are some product screenshots for Forge:

The 4 DORA metrics are only proxy metrics to real value that software development teams deliver to their organization and to their users. They are also only a subset to deeper outcomes that can be measured. Forge as a platform will hopefully be extended over time to allow teams to track more of what they care about.

Team members
Kee Wei Lam, Software Engineer
Tim Groleau, Software Engineer

Reference
[1] Cannot Measure Productivity
[2] State of DevOps Report