If you’re digging into engineering metrics for the first time, it can be hard to know what “good” looks like. The urge is strong to find a way to compare yourself to other companies using benchmarks, to know exactly where your teams sit on the distribution of excellent to terrible.
In reality, your best benchmark is your own baseline. Productivity, experience, and business metrics can vary greatly depending on a company’s size, age, and culture: your goal is continuous improvement, not trying to match companies you know little about.
Still, it’s useful to have a sense of what good might look like, so you can understand what’s possible and what’s concerning. To help you out, we’ve published software engineering benchmarks that allow you to form realistic ambitions for a set of core engineering effectiveness metrics:
We’ve seen other approaches to engineering benchmarks fall short for two reasons: the list of metrics is too long, making it unclear where to begin; and the benchmarks encourage improvement far past the point of diminishing returns. (For instance, let’s say the typical coding time on a task is two hours. You could seek to reduce this in order to meet someone’s idea of “elite,” but it’s only worth it if everything else in the path to production consistently takes less than two hours.)
When we set out to identify “benchmarks,” we didn’t want to cast a ridiculously wide net or suggest unreasonable targets. Instead, we wanted to focus specifically on metrics that are:
These core metrics tend to generate a substantial backlog for most productivity enhancement efforts. You’ll identify more specific areas for attention and measurement over time.
DORA has come up with their own categorization, which is based on a survey question about the code base the respondents are mostly working on. In our experience, that gives a biased view.
Some others claim a scientific methodology, where they end up sorting a dataset, only to claim that the optimal coding time is less than 15 minutes. Sure enough, that might be the case for single-commit changes, but it’s generally not a helpful recommendation. Additionally, the type of work (embedded software vs. cloud backend) and various other factors affect the usefulness of the recommendation.
For our benchmarks, we’ve hand-picked the numbers based on conversations and metrics from thousands of companies, ranging from small startups to enterprises with tens of thousands of employees. They are rule-of-thumb estimates, and in our experience, surprisingly accurate. Yet, it’s good to acknowledge that all engineering metrics are highly context-dependent, and the benchmarks may not all be relevant in your unique situation.
To provide clear-cut improvement guidance, we defined three levels of attainment for each of the benchmark metrics. These levels clarify where to concentrate your efforts for maximum impact and what can be set aside for the time being. Instead of adhering to strict percentile ranges, the focus is on identifying opportunities for driving step-change improvements.
Here’s what each level means in a bit more detail:
The exact path to improvement will always depend on your organization’s particulars, but there are some common culprits if you have metrics that are on the “needs attention” end of things. Here are some ideas for improving each metric:
Implementing these strategies requires focusing on the technical and cultural aspects of the engineering team and its processes. The tools and insights in Swarmia can support teams in identifying areas for improvement and tracking progress over time, aligning with the goal of continuous improvement.
Some of the benchmarks might seem absolutely unattainable to you today, and that’s OK. Swarmia gives you the tools to discover opportunities, implement changes, and see the results as they happen.
As a first step, gathering metrics for your team will help you identify the most critical bottlenecks in your process. Once you have your baseline metrics, Swarmia helps you set achievable goals for improvement based on your unique situation.
Subscribe to our newsletter
Get the latest product updates and #goodreads delivered to your inbox once a month.