How Miro delivers relevant insights to every level of the engineering organization

The Engineering Productivity team at Miro is dedicated to creating a data-driven engineering discipline and enabling leaders on all levels to optimize processes across the company’s globally distributed engineering organization. Miro’s engineering culture is based on autonomy with its freedom-within-a-frame principle, where each team has the right to make decisions within their control but are also offered guidance on what needs to be in place to glue the Miro way-of-working together.

The Engineering Productivity team was serving a set of metrics to Miro’s 100+ engineering teams. While some of the metrics representing feature adoption and quality were easier to surface, the team was struggling to provide reliable, up-to-date, meaningful, and sufficiently granular data on the speed of software development.

The Engineering Productivity team quickly realized that building dashboards of widely adopted yet continuously evolving metrics, like DORA and SPACE, was not worth the investment.

In addition to the cost of keeping the data in sync and the quality high, the engineering teams at Miro wanted a tool that would allow them to interact with the data to enable root cause analysis on potential improvement opportunities within essential engineering processes and allow teams to drive actual change. The existing dashboard was not offering this functionality, which was hindering concrete improvements. Digging deep into the data — from high-level trend analysis all the way to actual source datasets — was critical to the teams because it would give them the detail they needed to improve engineering process flows.

Furthermore, Miro’s existing tech stack didn’t offer a customizable notification system for engineers or teams. This resulted in a pull model, where engineers needed to build a habit of regularly checking in on key metrics rather than getting relevant data delivered to them.

Bringing unified data to everyone’s fingertips

Before starting the vendor selection process, the Engineering Productivity team worked with the teams to improve their Jira hygiene, rolled out unified naming conventions for key elements in the software development lifecycle, and increased data consistency on a global level. While it was important that the teams would continue to make their own decisions, the Engineering Productivity team set some ground rules for maintaining data quality going forward.

When Walter and his team were happy with the data quality improvements and convinced they wouldn’t face a garbage-in, garbage-out situation, they invited a pilot group of engineering managers, senior engineering managers, and a member of the Engineering Leadership team to meet with three shortlisted vendors.

After an introductory meeting and a demo with each vendor, they ran a survey with the pilot group, which resulted in Swarmia getting the highest outcome scores and the pilot group’s consensus that it would be able to address the key challenges Miro had identified. Most importantly, Swarmia allowed Miro to hold on to their freedom-within-a-frame principle without compromising data quality.

Powering internal dashboards with data from Swarmia

Around the same time Miro started trialing Swarmia, they also introduced an instrument called the Five Metrics. It consists of a combination of customer-centric metrics related to quality and support (bug resolution time and follow up on service requests), capacity spend (balance framework), and the speed of delivery (deployments and pull request cycle time).

However, instead of sending hundreds of engineers to separate platforms to find each of these metrics, Walter and his team consolidated the Five Metrics into their data lake that feeds it into a shared dashboard, giving everyone a single entry point into the most important, highest-level metrics. In this system, Swarmia acts as a key provider of engineering related metrics.

In addition to limiting the Miro engineers’ attention to the metrics that matter, the layered approach allows them to drill deeper into any specific areas that may interest them. On a high level, the reporting structure works as follows:

• Layer 1: A dashboard that acts as a single point of entry to the Five Metrics
• Layer 2: Supporting metrics for each of the Five Metrics to allow quick analysis in the dashboard (e.g. showing throughput side by side with pull request cycle time)
• Layer 3: Root cause analysis with links from the main dashboard to specific views in Swarmia, with the right filters as presets to save time and error

Balancing engineering investments

In a large engineering organization like Miro, it can be difficult to get visibility into where engineering capacity goes: Are all the strategic initiatives staffed appropriately? Are the product teams able to spend enough time on improving existing features? Are some teams struggling to keep the existing software operational?

Even though Miro already had a solution in place for tracking engineering investments before Swarmia, they appreciate Swarmia’s ability to customize the investment categories and create multiple ways to slice and dice the data.

Moving from a pull to a push model in delivering insights to engineers

As a part of their efforts to extend Miro’s data-driven engineering culture to every corner of the organization, the Engineering Productivity team wanted to start pushing relevant data to the teams rather than expecting them to spend time searching for the data they were interested in.

Teams were encouraged to adopt Swarmia’s working agreements and Slack notifications to introduce a push model. The idea is that teams set their own agreements and get both real-time and scheduled Slack notifications corresponding to the targets they’ve set for themselves.

Building trust in the data

After the first few months of using Swarmia, Walter could already sense a change in the way the teams were consuming and using data in their daily work.

The engineers at Miro knew from experience that building data pipelines takes a lot of effort, and failing to factor in important nuances can completely break trust in the data. However, the more they explored Swarmia and the dashboards it powered, the more they started relying on the insights it provided and taking advantage of it in their work.

In fact, what started out as a 25-person trial organically expanded into 100 engineers trying out the platform as they kept hearing about it from their coworkers. To further prove Swarmia’s value to everyone in the organization, the Miro Engineering Leadership team invited a software engineer to demo his real-life Swarmia use cases to them.

Walter, on the other hand, was relieved that the engineering analysts in his team no longer have to spend time on building and maintaining data pipelines for pulling commodity metrics like DORA into their dashboard. Instead, they can now get this data from Swarmia and spend time on more meaningful and value-adding activities, like data analysis.

Overall, Walter and the engineering teams at Miro are happy about the close partnership they’ve built with Swarmia and the speed at which they’re able to move things forward.

While the Engineering Productivity team acknowledges that completing the rollout of any new platform in such a large organization will take a while, they’ve been positively surprised about the speed of Swarmia adoption. Miro’s layered approach to reporting gives everyone in the engineering organization access to the data they need, whether it’s the high-level metrics in the dashboard or the more granular data in Swarmia.