How TrueLayer drives engineering effectiveness at every level of the organization

Maryanna Rayko leads TrueLayer’s 100-person product engineering organization, who are in charge of building and maintaining Europe’s leading open banking payments network. As the VP of Engineering, one of Maryanna’s main goals is to introduce more structure to the engineering organization so they can scale more effectively.

As a part of the initiative to improve efficiency across the engineering organization, Maryanna started rolling out three types of metrics:

• Product metrics
• Operational metrics
• Productivity metrics

While product and operational metrics were relatively easy to surface with a combination of internal systems and tools like OpsLevel, the productivity aspect turned out to be the trickiest to measure. TrueLayer had previously tried (the now-discontinued) Athenian but had abandoned it because it offered little to no transparency into how different metrics were calculated, resulting in the engineers and managers becoming reluctant to engage with the tool.

When it came to productivity, I was looking for something that would help me empower the teams to improve. I’m really keen for our engineers and managers to use the tool because I want to help the teams help themselves. I do need access to the data, but my main objective is that teams can improve their own productivity.

Maryanna Rayko
VP of Engineering

In addition to empowering teams, Maryanna was also interested in understanding the cost of KTLO (keeping the lights on) to make informed decisions about prioritizing and addressing tech debt as well as other KTLO reducing initiatives.

CTO Luca Martinetti, on the other hand, needed to measure engineering investment and understand the return on that investment. This became especially important as the engineering industry at large entered a cost-efficiency mode due to rising interest rates.

Like many other tech companies, we now operate with more constraints. We need to make engineering really count and choose what we invest in more carefully. At this stage, it’s important for us to get visibility into where teams are spending their time and whether they’re struggling somehow.

Luca Martinetti
CTO

Increasing visibility at every level of the engineering organization

Maryanna and the managers in the product engineering organization started looking into tools that would help solve their two key objectives: empowering teams to make better decisions and understanding the return on engineering investment.

From the first meeting with Swarmia, Maryanna understood that there was perfect philosophical alignment between how she thinks about the problem space and how Swarmia approaches solving those problems.

I immediately liked what I saw. Swarmia aligns with our way of thinking about the importance of empowering teams and checks all the boxes I have for a tool like this.

Maryanna Rayko
VP of Engineering

Measuring the return on engineering investment

For Maryanna, Luca, and the rest of the engineering leadership team, the investment balance view in Swarmia answers many of the questions they have about where teams are spending their time and whether there are opportunities to allocate engineering effort better.

From the start, it was important that the investment balance would reflect reality. We make a lot of decisions based on the data, so we need to be able to trust it.

Maryanna Rayko
VP of Engineering

The investment balance view gives the engineering leaders much-needed visibility into how much time different teams spend keeping the lights on. TrueLayer’s investment category setup in Swarmia also allows them to see the amount of the previous year’s feature work that has rolled over to the new year, which helps them improve planning and prioritization going forward.

The initiatives view offers another valuable lens into product development work, allowing engineering leaders to quickly see the status and progress of ongoing cross-team initiatives so they can proactively jump in and support the teams that may need it.

It’s possible that some teams have more workload than they can handle. When that happens, we want to be able to offer them help quickly — and Swarmia helps us do just that.

Luca Martinetti
CTO

Currently, the TrueLayer team is evaluating the feature and comparing it to Jira Advanced Roadmaps. While a wider rollout has yet to happen, many of the engineering managers already prefer Swarmia’s initiatives view as they find it easier to use than Jira’s corresponding solution.

Enabling managers to eliminate blockers

Silvia Di Nardo, a Senior Engineering Manager in TrueLayer’s product engineering organization, says Swarmia has quickly opened the doors to many valuable conversations in and between teams.

Swarmia allows us to see what we’re spending our time on. This makes it simpler for teams to understand the reasons behind project delays, leading to overall improvements in planning effectiveness.

Silvia Di Nardo
Senior Engineering Manager

Engineering Manager Michael Slack has found a lot of value in Swarmia’s working agreements, as they provide a solution to something he’s been struggling with as a leader: keeping the team accountable to the targets they set for themselves.

Working agreements allow us to see that we’re trending in the right direction on everything we consider important. The team likes being held accountable to the decisions we’re making.

Michael Slack
Engineering Manager

Michael’s team has also started to regularly review their work log in Swarmia to see all ongoing work at a glance, identify patterns, and assess their progress.

A big problem for my team is that we work with many legacy services. Swarmia has allowed us to prove that those legacy services influence our lead time and cycle time. Now, when we make bigger changes, we can easily prove their impact.

Michael Slack
Engineering Manager

Empowering engineers with better insights and automated feedback loops

While Maryanna and the engineering managers spend a lot of time in the Swarmia app, the engineers at TrueLayer primarily engage with the platform through its two-way Slack notifications. This way, the teams can automate some of the feedback loops that are crucial for their workflows.

Swarmia’s Slack notifications really drew the developers in. The unnecessary labor of going into Slack and saying, “can I get a review on this?” has gone away. I’ve heard a lot of positive feedback about that from the team.

Michael Slack
Engineering Manager

81% adoption in three months and unique value for everyone in the organizations

In the first three months of using Swarmia, TrueLayer has seen impressive adoption numbers, with 81% of the engineering organization fully onboarded. Over half of the engineers have also started using Swarmia’s Slack notifications. As a result, after the first quarter, 69% of the users rated their overall experience with Swarmia as either good or excellent.

Additionally, the teams in the product engineering organization have seen impressive productivity improvements over the first three months of actively using the tool. Namely, they’ve cut 18% of their cycle times, improved their already remarkable code review rate by 2%, and reduced time to first review by 30%.

What I really like about Swarmia is that everyone in the product engineering organization uses it in different ways. They all get value out of it, but that value changes based on their role.

Silvia Di Nardo
Senior Engineering Manager

With the product adoption already at an exceptional level, Silvia expects it to increase even further as they’re moving their Swarmia login to Okta, allowing even more people at TrueLayer to get into Swarmia.

We’re excited about the new Okta integration, as it allows all our product managers and other stakeholders who may not have GitHub credentials to access Swarmia.

Silvia Di Nardo
Senior Engineering Manager

The engineering leaders at TrueLayer are also looking forward to the launch of Swarmia’s developer experience surveys. Ultimately, it’ll allow them to correlate developer experience data and feedback from their engineers with the system metrics that are already available to them in Swarmia.