Untangling the Gordian Knot: A Guide to Overcoming Modern Product Development Challenges

#managing_others I explain how to untie the Gordian knot that plagues all would-be product development leaders

In ancient times, the legend of the Gordian knot perplexed many who attempted to unravel it. According to the story, if anyone managed to untangle the knot, they would become ruler of all Asia. Alexander the Great was one such aspirant, who famously solved this enigma by simply drawing his sword and cutting through the knot in one swift stroke.

Today's technology leaders face modern-day equivalents of the Gordian Knot - unrelenting pressures to build new features to grow the business while simultaneously addressing technical debt which is crippling developer velocity through poor productivity and a stream of unplanned work due to fragile code.

In this article, we’ll discuss how you too can unravel your Gordian knot - no sword required.

Understanding how we get here

In the early stages of a project, things are often straightforward and predictable. The user base is small, the business model is simple, and the technology stack is manageable. It's akin to a young sapling, easy to shape and guide in its growth.

However, as the business grows and the complexity of the solution increases, so too does the complexity of managing it. The sapling becomes a tree with branches spreading in all directions. Systems, processes, and platforms that were initially designed for a simpler time now need to support situations they were never intended for.

This evolution creates a situation where unplanned work begins to creep in. Bugs surface from legacy code, new features require more complex integrations than anticipated, or scalability issues arise as user numbers surge. Your user onboarding flow, which worked brilliantly before, can no longer drive the same acquisition numbers it once did. All of this drama, creates unplanned work which competes against planned work - new feature development or strategic initiatives - for resources.

Given that resources within any technology company are finite - primarily people - it becomes increasingly challenging to get things done on time without resorting to individual heroics. Deadlines become harder to meet as unplanned work competes against planned work for attention. Everyone starts to work very hard on good ideas. Your CEO may even announce multiple “pillars,” initiatives, tiger teams, and other focused efforts to encourage more activity and action. Strangely, all of this has a weird way of making things worse. The knot you were in - seems to get tighter.

Undoing the knot

It turns out there is a solution to this dilemma, but it does not come from computer science but from a theory in management science called the Theory of Constraints (TOC), popularized by its inventor Eliyahu M. Goldratt in his famous novel called The Goal.

Eliyahu observed that in many systems, and we use systems broadly here, including but not limited to manufacturing, software development, product experience, or any general process relating to people, rely on a sequence of steps that advance the goals so that ultimately something is defined as “done.”

In these systems, there could be many things that could be improved across each step from beginning to end, but the key insight Eliyahu highlights is that there is always one key constraint that is worse than any other improvement opportunity. If you address an improvement upstream of your constraint, you don’t improve the system because the key constraint still constrains you. If you happen to address an improvement that is downstream of your constraint, you still don’t improve the system for the same reason!

This is profound and it gives us a critical clue for how we can undo our Gordian knot in the software field: by identifying and addressing the key constraint, you will improve the system. Spreading your focus and your resources on any other constraint but the key constraint, doesn’t necessarily help you right now.

Applying TOC to our Gordian knot means identifying the most common source of all unplanned work and attacking it head-on. Is it technical debt from legacy code? Is it poor documentation leading to repeated errors? Or perhaps it's inadequate testing procedures leading to bugs slipping through? Perhaps it’s a single person in the loop of all of your processes who needs to review all of the work?

By identifying the constraint and applying focused effort toward untangling it, you can begin to regain control over your project's direction and pace.

Some years ago, when I was a manager at Movaris, I found myself in the middle of a crisis. We had haphazardly delivered software, installed in our customer’s data centers, that was exceptionally buggy. We had committed to the company that we were going to deliver new features next quarter that would help us grow our business, but the entire product and engineering team was constantly receiving escalations from customer support regarding critical blocking bugs that our customers were experiencing that needed to be addressed.

Though it was unpopular at the time, we spent time understanding where the source of the issues was and made the difficult decision to refactor and reduce technical debt in the buggiest module. This involved deallocating the best engineers who were most familiar with this area to address the problem, and it meant reneging on our commitments. However, with the dedicated effort, we delivered a patched version of our past release which consequently stemmed the tidal wave of bugs.

In fact, I’ve seen this lesson learned over and over again by teams in different ways. In my consulting practice, I’ve urged companies to stop optimizing their entire user acquisition funnel without addressing the key parts of the user experience where people abandon the sign-up process. I’ve had uncomfortable conversations with CEOs and helped them understand that by approving every change that happens inside their company, they won’t scale the company no matter how many engineers they have.

In our next section, we'll delve deeper into how you can identify your system’s key constraint and strategies for tackling it effectively.

Practical Suggestions to Identify Your Key Constraint

Perform Retrospectives on Projects

One of the most effective ways to identify constraints is by conducting retrospectives on completed projects. This involves analyzing how and why deadlines were missed or extended. For instance, if a project was delayed due to frequent code refactoring, it could indicate that technical debt is your Gordian knot. By addressing this constraint - perhaps through better coding practices or dedicated time for refactoring - you can improve the overall project delivery timeline.

Meet with Individual Contributors

Another approach is to have candid conversations with individual contributors. These are the people who are often at the coalface of your projects and can provide valuable insights into what impedes their work. For example, if developers consistently mention that they spend a significant amount of time debugging, it might suggest that inadequate testing procedures are your constraint. By improving these procedures, you can reduce debugging time and increase productivity.

Identify Critical Path Constraints

The critical path method (CPM) is a project modeling technique used to identify the sequence of tasks that directly affects the project completion date. Sometimes, the critical path is a person who has become a bottleneck due to an overload of responsibilities. Other times, it could be a process like code approval or deployment speed. Identifying these constraints and addressing them - perhaps by redistributing workload or automating certain processes - can significantly improve project timelines.

Use Surveys for Anonymous Feedback

Surveys provide an avenue for team members to share their thoughts anonymously, which can sometimes yield more honest feedback. If multiple respondents highlight issues such as poor documentation leading to repeated errors, it's likely that this is your constraint. Addressing this issue by improving documentation standards could lead to fewer errors and increased efficiency.

Measure End-to-End Project Delivery Time

Finally, measuring how long it takes to deliver a project from start to finish can help identify bottlenecks in your process. This includes measuring each step in the process separately to determine which steps are most time-consuming or expensive. For example, if integration testing consistently takes longer than expected, it may be your primary constraint. By optimizing this step - perhaps through better tooling or more efficient processes - you can reduce overall project delivery time.

Be Great

Just like Alexander the Great who cut through his Gordian Knot with a single stroke of his sword, you too have the power to cut through your management challenges with strategic actions based on understanding your constraints.

Remember that every complex system has at least one limiting factor that can significantly improve overall performance and productivity when identified and addressed. But here’s the rub: unlocking your key constraint will invariably reveal the next constraint to release. Knock enough of these out, and you’ll create real change and innovation.

I’ve observed over the years that great leaders, real change agents who help companies scale, are systematic in their pursuit of progress. By following this approach, you’ll blaze the fastest path for your organization to achieve greatness. Using any other approach will waste time and money and eventually you will lose credibility and trust. This approach won’t guarantee success because sometimes, some messes are so big, it takes time to fix, and you might not have the opportunity to experience the fruits of your labor, but at least you’ll know that you’re driving meaningful steps towards positive change.