As Director of Product & Engineering Operations at Smartly, Frederic Calvez oversees a broad range of responsibilities across the product development function — from DevOps and incident management to developer experience and tooling. With product development being Smartly’s largest function, having reliable data to drive decisions is critical.
When looking for ways to improve their engineering processes, the team at Smartly recognized they needed a solution that would give them better visibility into their development workflows without creating additional overhead.
The main reason my colleagues started looking into a tool like Swarmia — even before I started at Smartly in 2022 — was that they wanted to empower our product team and the engineering managers with a data-driven continuous improvement loop.
Director of Product & Engineering Operations
While Fred admits that technically, the product and engineering teams could’ve gathered data from Jira and GitHub on their own, the process was so manual and time consuming that only few of the teams considered it worth the effort.
Fred also has previous experience working at a company where similar data was available but only on a weekly basis. This was because the engineers would have to spend time on pulling the relevant metrics from their data warehouse, making the approach unscalable in the long term.
Delivering insights across the engineering organization
At Smartly, different teams and roles use Swarmia in distinct ways to improve their engineering processes and delivery. From tracking investment balance to managing sprints, the platform has become an essential tool for driving improvements across the organization.
Tracking engineering investments
One of the core use cases at Smartly is monitoring how engineering resources are allocated across different types of work, from maintaining the existing parts of the software to building new features.
Investment balance is something we check monthly. We want to be around 75 to 80 percent on roadmap items, and with Swarmia, we know we’re there.
Director of Product & Engineering Operations
Streamlining project delivery with initiatives
For Fred and his team, Swarmia’s initiatives feature has become an invaluable tool for project delivery and planning.
I check the initiatives view on a daily basis, really looking at how the plan is evolving, what the forecast looks like, how it’s projecting, and where the bottlenecks are. For me, it’s a good way to aggregate thousands of tickets in one single view without losing the ability to jump into specific subtasks when I need to.
Director of Product & Engineering Operations
This possibility to quickly zoom in from a bird’s-eye-view into the details allows engineering leaders to quickly identify and address bottlenecks without micromanaging. Instead, the leaders at Smartly can proactively offer their help to unblock teams and projects — without delay or unnecessarily involving too many people.
Improving cycle time and code review with working agreements and Slack notifications
One of the most immediate benefits of Swarmia for Smartly has been the reduction in pull request cycle times across all teams. Rather than chasing a specific number, Fred really appreciates the conversations any notable changes in the metrics encourage in the teams.
Just this week, I got an alert saying ‘Hey guys, somehow the overall group cycle time is tanking — what’s happening?’ I know we had a freeze during Black Friday, December was a big push for people to go on holiday, and now we start the new year with planning. Instead of coding, they’re doing more tech planning. We can clearly see that in the cycle time.
Director of Product & Engineering Operations
Smartly processes around $10 million in ad spend through their platform daily, making code quality crucial. As a result, they’ve adopted a strict zero-bug policy and a two-person review process. While the compliance requirements aren’t as stringent as a bank’s, the amount of money that moves through the platform makes the teams take every change they push into production very seriously. To maintain this standard, they rely heavily on Swarmia’s Slack integration.
The working agreements and Slack notifications are super useful. They give our engineers the nudges they need when they’re juggling different priorities.
Director of Product & Engineering Operations
Monitoring build times and CI costs
As part of their quality initiatives, Smartly has been expanding their automated test suite, particularly focusing on integration tests. Swarmia helps them keep their CI costs and build times in check.
Swarmia helps keep our build times in check so our CI costs don’t explode.
Director of Product & Engineering Operations
Implementing sprints and story points
While not all teams at Smartly use the same development methodology, they’re increasingly adopting sprints and story points. This is where Swarmia’s sprint view comes in handy.
We really want the teams to define their ways of working. We don’t mind too much if they choose Scrum or Kanban, but we’ve noticed that getting together to estimate story points is a great way for engineers to challenge themselves and each other. Having those conversations early encourages them to think ahead and work together on planning some of the more complicated stories.
Director of Product & Engineering Operations
Building a data-driven engineering culture
For Smartly, the impact of using Swarmia goes beyond just improving specific metrics. The platform has helped create a more data-driven engineering culture where teams are empowered to drive their own improvement. Rather than spending time manually collecting metrics, they can focus on using data to drive change.
We’ve built a self-aware engineering culture, where each team can see how they’re doing. What I really like about Swarmia is that it acts as a mirror for the teams. They use it to proactively identify opportunities for improvement without anyone having to tell them what to do.
Director of Product & Engineering Operations
Today, Swarmia serves as an integral part of how Smartly’s engineering organization operates. Through intuitive dashboards and detailed analytics, the platform helps everyone from individual contributors to engineering leadership access the insights they need to drive continuous improvement.
We can aggregate all this data at different levels: teams, groups, functions. Swarmia allows us to break down our data at any moment and at any level, which is something we wouldn’t be able to do if we had to crunch the numbers manually.
Director of Product & Engineering Operations
Swarmia has seen strong organic adoption across Smartly’s engineering organization. The platform’s Slack notifications and user-friendly interface have made it a natural part of engineers’ daily workflows.
The partnership between Smartly and Swarmia continues to evolve, with new features being rolled out frequently.
I’m excited by the features that are still in beta. It’s good to see that the product is evolving nicely.
Director of Product & Engineering Operations
