Honest Conversation with an Agile Purist – Part 1

Oh boy, does this subject line raise the level of emotion. I’m sure it does for lots of people. Lets face it, the industry has been doing Agile for over 20 years now, but the perfect world of Agile as envisioned in its ideal form is like a Unicorn. Perfect in every way and as unreal as the Easter Bunny.

The reason for this is that Agile in its perfect form is a discipline that collides with the principles of another discipline – corporate management. The large corporate world is where Agile software development seems to hit its most significant roadblocks. After all, a small company or a startup has a much smaller chain of command and can transform the way it wants to approach project management. It can be quite agile. A large org on the other hand, with many different levels in the chain of command, will face typical organization challenges.

The Agile Purist definitely says “….corporate environment or not, it can be done… just need to follow the Agile rules and adopt Agile mindset at the very top”. In the practical world, however, the cultural match of these two disciplines will be unlikely. But there is an opportunity to align towards a common goal. 

Imagine you’re the CEO of a large, publicly traded company. The Board has a plan it would like you to achieve. This plan has a goal, timeline, and is supported by financial projections. The goal is usually based on the revenue or profitability expectation and is supported by the financial budget required to get to the goal. The formula in its simplest form shows “investing X into an effort, results in Y expected increase in revenue, reduction of cost, or both”.

So let’s assume that this company has a goal that can be realized by implementing a technology solution. You, the CEO, have a conversation with your CTO and state the following: “The Board approved X million dollars towards this project, and we need the Technology team to execute this project by a specific date”. Assuming the conversations for evaluating whether the investment is adequate and timeline is reasonable have already been made and agreed upon even before the pitch to the Board, after approval the Project needs to start.

With a little bit of planning, tee-shirt sizing the Epics, and developing an overall logical estimation to the effort (hopefully before the project is kicked off) and the allocation of workload among one or many teams, the endeavor begins. The first thing that the project will be measured on is the ability to complete the project by a certain date. During the planning phase, what happens naturally by all of the senior leaders is that any original estimates and timeline projections provided by the team would be buffered by a certain coefficient both in terms of timelines and in terms of financial budget. Afterall, if this isn’t someone’s first rodeo of running a software project, chances are the project will run late and exceed the budget. Industry statistics show that projects typically experience 27% overrun of costs with 1 in each 6 projects exceeding 200%. So immediately, the senior leadership attitude is negative and skeptical. The senior leaders know they need to run this project to get to the next level, and probably have some personal skin in the game in the form of financial incentives and bonuses. Yet, the leaders will likely have serious doubts over the ability of the engineering teams to deliver on their promises even with padded timelines.

These are natural behaviors, and careers are made or broken based on the ability to deliver on the plan. In the midst of this, Agile Purist sticks to the mantra “we take one Sprint at a time, focus on the value, and discover along the way”. Sure, sounds good, but this type of answer doesn’t address the senior leadership’s ability to track the project’s progress and evaluate it on very simple criteria. Will it be done on time and on budget? And, if the plans run over time and budget expectations, then by how much? How soon would we find out?

It is important to note that small scale projects with one or two teams are easier to manage, make less mistakes in projections and forecasts, and can quickly adapt to realign priorities to deliver on time. However, on a larger the project, more teams and their interdependencies rely heavily on intra-team collaboration, communication, and other complexities, which all have an impact on the overall timeline and budget.

The corporate management often culture struggles evaluating the risks of the delivery by simply not being able to tell precisely the current phase of the project due to Agile methodology iterative discovery based approach to development. In Agile, the plan is very rough and new things are constantly being discovered, which adds more work extending the tail of the timeline. How many things will be discovered during the development is unknown, while senior leadership is asked to “trust” the team’s ability to solve any newly discovered challenges that require additional work. And, not simply trust, but trust it can be solved in due time.

Trust is hard to earn. Most senior leaders have lots of horror stories to share from their experience running previous projects, and most of those are likely to be some sort of a failure. The industry states that 66% of all  projects are deemed a failure, so the attitude here is not surprising. In addition it’s important to note that critical delays in software development typically are not discovered until the very end, when the project is due and very little can be done to change the course. 

So given this culture mismatch, what should we do? Well, the typical trend is for Agile Purist to educate the senior leadership in “the way of the Jedi”. After all, shouldn’t we educate senior leaders of the new modern, better, cooler and Agile way to develop software? If this approach succeeds and the leaders of the company begin to think differently while somehow managing to transpose Agile Discovery into elastically adjusted plans by partnering with the Board, Customers, and Sales, that’s amazing. But, let’s be realistic. Chances are that this won’t work at the large enterprise level.

Good news, there is another way…

Honest Conversation with an Agile Purist – Part 2