It’s table stakes – today’s product managers are expected to be fluent in quantitative data. You have user analytics for breakfast. You talk in MAUs, funnels, cohorts, and segments. For lunch, you estimate your product‘s impact on revenue and/or operating costs. What’s for dinner? OKRs.

Notice any blind spots?

Product managers are supposedly responsible for making sure their teams build the right thing, but few follow numbers on how they build it. This valuable information (what we call engineering intelligence data) often gets overlooked, to the detriment of the product.

But I hear you: this sounds like yet another thing to add to your bottomless list of product manager duties.

In fact, it‘s quite the contrary. I‘d go as far as to say that paying attention to this data makes your current tasks easier.

So in this article, I’ll explain how I use engineering intelligence data from Swarmia to:

• Maintain a fast product development machine
• Get visibility into engineers’ work
• Create better estimates and forecasts
• Prioritize all kinds of work while keeping stakeholders happy

Less ebb, more flow

One of product managers’ main responsibilities is to ensure their team preemptively validates the value, usability, technical feasibility, and business viability of the things they plan to build.

But once you’ve done the product discovery, you can’t just hand the spec to the engineers and lean back. In practice, product discovery doesn’t end when the implementation starts. “No plan survives contact with the enemy”, as Field Marshal Helmuth von Moltke wrote in 1871. (I hope you don’t regard your customers as enemies, but you get the point.)

So, instead of releasing everything in a big bang, you ship stuff in the smallest valuable increments. This way, you learn (and often fail) faster, your customers get value earlier, you can change plans nimbly, and you minimize risk. Product managers have a big role in splitting and sequencing the work since they should have a good idea of the minimum viable product — and what can be introduced later.

Yeah, yeah. You know the theory. But how do you keep track of that in practice? For me, I find it helps to pay attention to these metrics:

• Cycle time. This number tells if we’re shipping in small enough increments. We try to keep stories below two weeks and epics below two months. Every story should introduce customer-facing value.
• Work in progress. Working on fewer things at once makes our cycle times faster and reduces the burden of multitasking. We avoid working on more than one story per two engineers at a time.
• Flow efficiency. In-progress work sitting idly can be a symptom of interruptions or too sudden priority changes. We aim to work actively on an issue on >95% of its lifetime days.
• Scope creep. Sure, reacting to what we learn is more important than sticking to the plan. But ballooning stories and epics can be a sign of poor product discovery, inadequate planning, or that we’re too lax with scoping. A common anti-pattern is when engineers anxiously pack every conceivable task into the current story because we always hurry to move to the next thing. Something might be wrong if the number of subtasks in a story grows more than 50% during its lifetime (or is higher than 20 to begin with).

I can see these metrics in Swarmia’s flow insights.

Clearing the fog of work

Whoa, aren’t we now getting too much into the details of software delivery? Isn’t that the engineering manager’s responsibility?

It is, but it pays off for product managers to stay in the loop too. Agile product development is not a sequential process but a constant interplay between the problem, the solution, and the implementation. When I have visibility into the engineers’ work, I can:

• Detect delays and blockers early to make the necessary adjustments
• See who to talk to when I have a question or a last-minute suggestion
• See the unplanned work happening outside the roadmap
• Notice if the team lacks focus and help re-establish it

I can see some of these in Jira but to a limited degree. The issue tracker gives me a snapshot of the current situation, but the time dimension is mostly absent. This makes it harder to assess how the work is flowing. Also, the issues visible in Jira represent just the subset of work that gets logged there. Our Jira hygiene is not perfect, nor should it be. Sometimes we just want to ship things like small fixes to the customer as quickly as possible without the issue tracker getting in the way.

I use Swarmia’s work log to see what the team is really working on, with the Jira and pull request activities hand-in-hand on a timeline. The view also includes unlinked pull requests and can be grouped by issue, team member, repository, or investment category (more on that shortly).

When I want to zoom out a bit, I switch to the high-level work log, which shows one year’s worth of activity in a condensed format. Here, it’s easy to see the stories and epics that dragged on or lacked focus.

Swarmia’s initiatives show the same data from a different perspective. I go here when I want to see how an individual project is progressing. Initiatives aggregate data from multiple teams, repositories, and Jira projects onto the same timeline. I can move across the hierarchy from epics to stories to subtasks. It’s easy to see if there are gaps in activity or if we‘ve steadily moved from one increment to the next.

The initiatives view is my go-to for communicating the status of a project. It’s also pretty useful in retrospectives when looking back to see what actually happened instead of relying on vague memories or painstakingly building a timeline.

Predictions: impossible but necessary

At Swarmia, we try to avoid deadlines. We don’t want to sacrifice quality or people’s well-being, so we prefer not to promise fixed-scope features on a fixed date. Sometimes you must make high-integrity commitments for one reason or another, but that’s not our standard practice.

Yet, even without deadlines, we want to estimate the effort our work is going to take.

Firstly, planning things like cross-team initiatives and marketing launches becomes easier with more predictability in delivery timelines. And secondly, it’s not exactly a product management secret that you should work on the things that get you the most value with the least effort. Adding dark mode to your app might be worth it if you can do it in a day, but it could be out of the question if it takes a month.

Estimating effort is notoriously tricky, and few engineers particularly like it. There are many ways to approach estimation, but they all rely on drawing comparisons to previous work you‘ve done.

So, how well do you know the amount of effort your last story took? You can see the start and end dates from your issue tracker, but that’s just the calendar time. The engineers probably also had other issues in progress, worked on untracked ad-hoc pull requests, helped other teams, took sick leave, and so on. You get the picture.

Swarmia models how many full-time equivalent (FTE) person-months each item of work took based on engineers’ activities on issues, commits, and pull requests, as well as time-off information from our HR system. To see how you can use the effort data in forecasts, consider this simple example:

If you estimate in story points, perhaps you can dedicate a part of your next sprint retrospective to checking how well they correspond to FTE months. Based on that, you might notice you overestimate certain types of work while underestimating others.

Not only does the FTE model improve the accuracy of estimates, but it also acts as a single source of truth. This way, we avoid the unnecessary divergence of vibe-based judgments.

The effort numbers are visible all across the Swarmia app. If you look carefully, you can see them marked with a 👤 icon in all the screenshots in this article.

The anatomy of effort

Prioritization is at the core of a product manager’s job, but it can be a drag.

Salespeople say they can’t sell your product because it’s missing custom dashboards, on-premise hosting, and Microsoft Teams integration. Customer support is drowning in usability issues from data import, filter setup, and notification settings. Engineers think decaying legacy code, delayed dependency updates, and low test coverage are pushing the app to the brink of collapse.

Besides those, when was the last time you experimented with new technologies and moonshot ideas?

Maybe you could create a magic formula with 17 dimensions to tell you exactly what to work on next. You just need to make it sophisticated enough to address trade-offs between new features, usability, and technical debt. And you must carefully adjust the weights to hit the right balance between immediate needs, risky bets, and long-term sustainability. If you manage to pull that off, see how people react when you tell them you can’t work on the custom dashboards right now because it has only a score of 137.

A simpler solution is to decide on a strategic allocation between the different categories of work.

We track this in Swarmia’s investment balance using something called the “balance framework”:

• New things
• Improvements
• Productivity
• Keeping the lights on (KTLO)

We aim to spend less than 10% of our effort on KTLO, around 15% on productivity, and more than 60% on new things and improvements. It’s much easier to prioritize within each category since the goals and metrics are similar.

Prioritization isn’t just an intellectual decision but a communication challenge. In many teams, there’s tension between product managers, who want to move faster with the roadmap, and engineers, who’d like to address technical debt more. That happens despite it being kind of obvious to everyone that the business both needs new things to stay afloat and that skipping refactoring will come back haunting later.

A shared truth helps avoid unnecessary disagreements within and outside the team. Thanks to investment balance, I can debate less about how things are and focus more on how they ought to be.

Another part of the challenge is to align multiple teams toward the same goals to get maximum leverage as a business. No matter how well you prioritize stuff within your team, you’ll have a bad time if you do it in a vacuum. You don’t want to end up doing things like building on a soon-to-be-deprecated API or targeting a customer segment your company isn’t focusing on.

That’s where linking work to company objectives comes in.

In focus summary, I can see at a glance how my team’s effort was divided between different initiatives and issues during any given period. I use this view to check how our team’s investments align with the business goals and focus areas we’ve set.

Last year, one of our aims was to expand our developer experience offering and create a standalone pricing plan for it. With this goal in mind, my team spent 54% of our effort on the new developer experience surveys feature.

When we judge competing priorities, we look at our effort allocation and ask ourselves: “Is this taking us closer to our goals?” And again, having the data at hand makes it easier to explain the decisions.

Your move

That’s how I use Swarmia. I don’t know if all of that is applicable to your product and organization, but you might want to give engineering intelligence metrics a shot. Maybe your company already has a platform for that.

If not, it might happen soon. Gartner predicts that the share of software organizations using engineering intelligence platforms will rise from 5% in 2024 to 50% in 2027. People probably expect you to handle this stuff sooner or later.

Now is a great time to get a head start. Expect cycle times, FTEs, and investment categories to become a popular afternoon snack for product managers.

Build better products with Swarmia

Get a live demo with one of our product experts to understand how you can use Swarmia to build better products.

Get a demo