Boosting visibility, transparency, and engineering productivity at Prima

Tatiana Pipino is a Technical Program Manager at Prima. As a part of her role, she looks after the effectiveness of the whole engineering organization. Paolo D’Incau is a Senior Engineering Manager who is currently leading three of Prima’s engineering teams across one business domain.

When the Prima engineering organization first started looking into solutions for measuring engineering productivity, their goal was to improve visibility into organization-level metrics. The lack of visibility was making it difficult to recognize process bottlenecks and possible improvement areas.

At the company level, some engineering teams were using their issue tracker to build graphs, but those didn’t include any data from GitHub.

Tatiana Pipino
Technical Program Manager

Additionally, Paolo and the other engineering managers were interested in measuring DORA metrics and other healthy productivity measures, including work-in-progress.

I’d been reading a lot about DORA metrics and the SPACE framework. We’re always looking to improve the way we deliver software, and wanted to start measuring how effectively we were getting changes through, how many PRs we had in progress, and how often something breaks in production.

Paolo D’Incau
Senior Engineering Manager

It was important for Paolo that the tool they’d choose wouldn’t just be a dashboard for the engineering leadership. While he acknowledges that it is mainly the engineering managers’ job to collect metrics and bring discussion topics to the team, it’s up to the team to decide what — if anything — they’d like to do based on the data.

I think it’s very important to involve developers in discussions about metrics. They need to be able to access the metrics and analyze the numbers on their own, not just with their manager. At the end of the day, this data can also make our engineers happier because it allows them to clearly see the progress they’ve made.

Paolo D’Incau
Senior Engineering Manager

Better ways to track technical objectives and practical tools for continuous improvement

Choosing a tool for tracking DORA and other engineering metrics quickly became a critical project at Prima, as the company is known for its focus on technology and data. In addition to finding a partner that was aligned with Prima’s values, the goal was to arrive at a consistent measurement framework across teams and domains.

We did a lot of research online and had a group of people across the engineering organization evaluate different solutions. Ultimately, the decision to go with Swarmia came down to the data, simplicity, and user experience of the platform, which are all very important factors to Prima.

Paolo D’Incau
Senior Engineering Manager

Tatiana, who wasn’t directly involved in the evaluation process, has since become a power user of Swarmia. She immediately saw the value in using a single platform for tracking key metrics, such as DORA, across the organization.

Before Swarmia, there was no way for us to track engineering metrics at the company level. We had a lot of trouble with setting up some of the other tools. Unlike them, Swarmia was very easy to configure and start using.

Tatiana Pipino
Technical Program Manager

Paolo and the other engineering managers, on the other hand, have been using Swarmia as a tool for learning and sharing — not only within teams but also across them. They never use Swarmia to directly compare teams, but rather, to run experiments and report back on the results to the rest of the group to spark conversations and drive continuous improvement.

We have monthly discussions with the other domain leads and the CTO to exchange ideas and share the lessons we’ve learned. We talk about the improvements we’ve seen from Swarmia as well as some of the other tools we’re using.

Paolo D’Incau
Senior Engineering Manager

The engineering managers at Prima have found Swarmia’s working agreements and Slack notifications to be extremely valuable. The working agreements are particularly useful every time new teams are established in the growing engineering organization.

The use of working agreements, coupled with the monitoring of software development lifecycle metrics allows our new teams to experiment with different agreements until they find the ways of working that best suit their needs.

Tatiana Pipino
Technical Program Manager

More than anything, Paolo appreciates the conversations that were spurred by adopting and adjusting the working agreements in his teams.

The targets are great, but what was even better were the discussions we were able to have in the team when we were setting them. For example, I lead one team of six people, and since we do pair programming, we decided together that it was better for us to limit our work in progress to 3+1 pull requests to encourage collaboration than to say that we could have six PRs open at once.

Paolo D’Incau
Senior Engineering Manager

Huge leaps forward in productivity and cross-department collaboration

After using Swarmia for over a year, the Prima engineering organization has experienced massive productivity improvements across all engineering teams and domains.

In the past year, we’ve seen huge improvements in our ways of working as well as all our engineering metrics. We’re really happy with the progress we’ve made.

Tatiana Pipino
Technical Program Manager

With the baseline Tatiana now gets from Swarmia, it’s also easier for her and the teams to set more relevant engineering objectives going forward.

Paolo has been particularly impressed with the ease of analyzing and actioning data from Swarmia. Instead of chasing after individual metrics, he uses Swarmia daily to look at longer-term trends and find discussion starters that have the potential to drive the whole organization forward.

The coolest thing about Swarmia is that it gives me really useful insights. For example, we noticed a strange correlation between long-lasting pull requests and production incidents. Off the back of that realization, we were able to have great conversations between the engineering, product, and business teams and decide to start limiting the scope of our stories and PRs.

Paolo D’Incau
Senior Engineering Manager

Overall, both Tatiana and Paolo feel like Swarmia has provided them with the visibility and insights the whole engineering organization needs to continue to do their best work. As the business and its engineering organization keep growing, the data-driven culture they’ve built over the past year will allow them to scale more effectively.