Once a software engineering organization has a few teams, engineering leaders are called upon to manage projects that span those teams. It’s not just about keeping tabs on what each team is up to; it’s about understanding how all the different pieces fit together. Effective leaders must have the full picture: who’s doing what, how tasks and dependencies line up, and how all this effort pushes the project’s objectives forward.
It sounds hard because it is. Many companies don’t have a one-size-fits-all method for handling these types of projects and end up winging it each time a new project comes along, resorting to manual, error-prone, process-dependent techniques to keep track of everything every time. In this blog post, I’ll discuss why this is such a tricky problem, some proven tactics for keeping big projects moving, and how our new initiatives feature aims to make things smoother.
Where spreadsheets and issue trackers fall short
One common method for tracking initiatives involves a bespoke process supported by a colorful spreadsheet that is updated manually on some cadence.
At one 1,000+ engineer organization I worked with, cross-team initiatives for a large corner of the organization were managed in a spreadsheet like this. There was a row for every ongoing initiative and a column for each week where the owner would write an update and color-code their understanding of the initiative’s status. All updates were manual, and there was no attempt to connect the spreadsheet with the actual work being done — there wasn’t even a link to a ticket or other artifact representing the effort.
Dozens of very expensive leaders sat in an hour-long weekly meeting to report on any risks to completing an initiative, and their reports were reported up the management chain. More than a year into one massive engineering-wide effort, leaders had to acknowledge that the spreadsheet’s representation of reality was far from the truth, and the initiative that had been mostly “green” for months was a massive failure.
Alternatively, some teams turn to their issue trackers, setting up a network of interrelated tickets, stories, epics, and tasks, organized using a variety of labels and custom fields. While this uses an existing tool instead of returning to a spreadsheet, it creates a tangled web of tickets. Tickets are great for guiding execution but won’t show a current view of the initiative’s health without careful tending.
Trying to change course with estimates and deadlines
Far too many organizations turn to estimation to try to make these problems go away, attempting to make an educated quantitative guess about the difficulty of each story and then allowing a team to only work on a certain amount of estimated difficulty (or story points) in a period. The theory is that you can math your way to a plausible delivery date given estimates and team capacity. The reality is that effective software development rarely works this way, and time spent on estimation isn’t likely to solve predictability challenges.
When teams try to conquer these problems via estimation, it’s easy to end up with a labor-intensive, error-prone process that provides a false sense of precision. The health of a project is often assessed based on gut feelings and past commitments rather than hard data, which is precarious when strategic decisions need to be made. If issue tracker data isn’t deliberately and continuously synchronized with other parts of the development process, it quickly becomes outdated. If it’s not easy to check the plan against reality, reality can quickly get out of hand.
Perhaps most importantly, though, these makeshift methods rarely offer insights into a critical question that usually feels too complicated to ask: Was the outcome of the initiative worth the time spent? That’s one of many problems solved by our new Initiatives feature.
Toward predictable, transparent delivery
Successfully and predictably leading complex, cross-cutting initiatives in a software engineering organization requires timely, accurate, trustworthy data. With that knowledge in hand, you can ensure that progress is being made with a reasonable scope and a reasonable amount of engineering resources.
If things aren’t moving along as quickly as you’d hope, there are a few common culprits you can look for and address:
- Doing too many things at once: When teams try to handle too many tasks simultaneously, it leads to interruptions and context switching, drastically reducing productivity and focus. Team members become overwhelmed, leading to a decrease in the quality of work and delays in project timelines. Teams need to prioritize tasks, define specific focus areas for the team, and implement work-in-progress limits; advocate for realistic planning based on the team’s scope and obligations. Leaders should limit the number of initiatives a team is expected to work on at any given time.
- Working on increments that are too large: Large increments can extend development cycles, reducing the team’s ability to adapt to changes and delaying feedback. This approach can also overwhelm the team and make it challenging to track progress. Teams should break down work into small increments — tasks that can be completed in one or two days. Smaller increments allow for quicker feedback, easier adjustments, and a clearer demonstration of progress. Small increments are also proven to increase overall throughput.
- Relying on individuals vs. the team: When an initiative depends excessively on a single person, bottlenecks and delays arise when those individuals are overloaded or unavailable. The pattern also undermines team collaboration and knowledge sharing. Leaders at every level must encourage a team-oriented approach where knowledge and responsibilities are shared. Incorporate cross-training and collaborative work practices to ensure the team can make progress even when key individuals are absent.
- Failing to incorporate new information: When a team sticks too rigidly to a plan without adapting to new information or changing circumstances, you end up with outdated solutions and missed opportunities. Leaders must promote and cultivate a flexible mindset, encouraging teams to revisit and revise plans as new information becomes available.
- Focusing on outputs over outcomes: When initiatives are evaluated solely on outputs (like the number of story points or features completed), it’s easy to lose sight of the actual goals of the initiative, such as improving user satisfaction or increasing sales. This misalignment can lead to inefficiencies and time spent on work that doesn’t contribute to the objective. Focus instead on the outcomes the project is trying to achieve. Set clear (preferably user/customer-centric) goals, and measure progress toward them to ensure that work aligns with the project’s desired outcome(s).
- Ignoring hidden work and “keeping the lights on” work: Often, there’s significant work involved in maintaining existing systems, which goes unnoticed or underestimated. Ignoring this aspect can strain resources and impact the delivery of new projects. Account for maintenance and operational work when planning initiatives, and adjust your expectations as needed as the initiative proceeds.
How Swarmia helps
The Initiatives feature uses trustworthy data from the software development lifecycle to help you spot these sorts of problems before they get out of hand, starting with a bird’s-eye view of what your teams are working on and where they might be stuck.
Consider a software company implementing a new payment system involving various teams like backend, frontend, and data tooling. Swarmia’s Initiatives feature tracks each team’s contributions. It automatically integrates updates from coding activities and issue trackers, providing real-time data and visualizations and eliminating the need for constant meetings to assess progress.
If there are scope changes, for example due to customer feedback or new regulatory requirements, Swarmia highlights these, allowing leaders to quickly evaluate the impact on the project’s timeline and resources.
We also make it easy to see which teams are contributing and how much.
When an initiative is completed, Swarmia provides insights into the initiative’s success and the appropriateness of the efforts and resources expended relative to the impact of the work.
By providing leaders with real-time data, reducing manual work, and allowing for proactive management, Swarmia ensures that initiatives are executed effectively and align with business goals. Leaders can move from uncertainty and guesswork to clarity and strategy, making informed decisions that drive successful outcomes.
