Recently, the company where I work has migrated completely to an iterative Agile approach to software development. Although it took some effort to get everyone on the same page internally, the migration has been a success so far. Our two professional teams are doing a marvelous job with their product owners. Quality has improved, team spirit is higher than ever and functionality is being delivered steadily every sprint. We even have a team of four interns that are using Scrum successfully for their internship projects.
But selling it...
Despite the internal success, we find it very hard to sell Scrum (or the Agile philosophy in general) to our customers. Most customers appreciate the focus on transparency, business value, customer involvement and change. They understand Scrum and they like it. But when all is said and done, they still want to know how much a given set of functionality is going to cost them, what will be implemented and when it will be ready. In short, they want a level of guarantees that I feel are very hard to give (in any situation, Agile or not) but are understandable.
When you’re using an Agile approach to software development, you are acknowledging that change in scope will happen. Because you can’t put a fixed price or a fixed deadline on a moving target, the most ‘Agile’ way to work with a customer is to sell sprints to deliver functionality from a product backlog. As changes occur, the product backlog is adjusted and the product owner can determine if additional sprints will be necessary based on the point estimates and the development team’s velocity.
If it were that easy ….
'I prefer the old way'
One customer actually lamented that he preferred the traditional way of doing things, which boiled down to writing a massive (60+) page specification document, estimating hours based on that and translating it to a fixed price, picking a deadline and getting to work. And I can totally understand this. From the customer’s perspective, this gave them the guarantees on budget, date and scope they needed to sell the project internally to their own management, which demanded similar guarantees.
The irony is that, despite these guarantees, the scope for these projects often changed during the project. So we had to move deadlines, requested additional budget for new functionality and frequently had to cover overly optimistic hour estimates from our internal budgets. We had a lot of difficult sessions with customers to change the deadline and/or increase the budget. But the bottom-line is that it usually did happen. So the initial guarantees proved to be false anyways. But they provided a sense of certainty and security to the customer that was needed to sell the project. Once underway, the customer often had little choice but to increase budget and move deadline as they had committed.
Although I am a strong proponent of Agile methodologies and Scrum in particular, I feel that there is a lot of work to do when it comes to figuring out how to sell our ‘shiny Agile approaches to software development’. Although an Agile approach seems to be the most rational approach, and customers usually agree with that, there is still a strong desire to get guarantees on price, scope and schedule before the project begins.
Why customers want guarantees on scope, budget and schedule
An important question to ask is why customers want to have a guarantee on scope, budget and schedule in the first place? Put yourself in their shoes for a minute. Suppose you are buying a house and want to renovate the kitchen and the bathroom. You wouldn’t be happy either when the construction company does not want to commit to a budget, a scope and a schedule. It may feel as if you have to shoulder the full risk of the project. If the construction work takes more time, you pay and have to wait longer before you can use your house. If the construction work uncovers issues (like leakage), you pay. This comparison is actually false, and I will come back to it later. But a lot of customers reason like this, with these kinds of examples.
In short, guarantees on scope, budget and schedule mitigate risk for the customer. Use the above example to reflect on how you would respond in their shoes to an Agile proposal. So how do you sell your Agile project with this in mind? Let’s take a look at three things I learned in the past few months that might help.
Advice 1: Don’t sell the method, sell the advantages
One mistake that we made at first, was that we tried to sell a methodology (Scrum) rather than a set of advantages. I strongly believe that an iterative (Agile) approach is the single best way to develop software. But you have to explain very clearly why it works and that the methodology is just a means of getting there. Explaining all the Scrum meetings and the rules is not going to help. Focus on what you try to achieve with an Agile approach, which is:
- Focus on change: Agile development emphasizes that change in scope is part of software development. Changes in scope can have many reasons, including (amongst others) evolving insights in how a system should work or how a process should be automated, new ideas that pop up during the process, external factors (economic, political, judicial, etc.), because of complexities or underestimates, and because of user feedback. Customers should see the value of an approach that allows for change. Too many projects fail because the project delivered the system that was designed, but not the system that was needed;
- Focus on iterative development: Agile development emphasizes that the best way to deal with the complexities of software development, is to work in small steps towards a fairly vague end state rather than trying to nail down all the details from the start. The details will change anyways, as new insights are gained. This avoids costly functional and technical designs that are hard to read, open to many interpretations and will only result in contract negotiation as heated debates ensue over what constitutes a certain functionality. Customers should see the benefit in being able to start as soon as possible, rather than having to wait for massive design documents (which are hard to read anyways);
- Focus on frequent feedback cycles: Agile development emphasizes that for change to be identified, frequent feedback cycles are necessary. In Scrum, this is done through the daily scrum, the sprint review and the retrospective. Customers should welcome this, as it allows them to incorporate changing insights;
- Focus on delivering functionality to achieve business value: Agile development emphasizes on delivery of functionality to achieve business value at the end of every iteration or sprint. Instead of having to wait for months, customers will be able to inspect progress periodically. Even better, each sprint can potentially deliver functionality that can be put into production or used to convince higher management or end users that the project is really going somewhere;
- Focus on removing the black box: Agile development emphasizes that the best way to develop software, is to put the customer in the room with the people that are actually going to build the system. This allows for the most direct communication and sharing of knowledge as possible. Customers should welcome this as it opens the black box that software development is.
The bottom line is that when you are using an Agile approach to software development, you acknowledge that change will occur and should be welcomed and even actively sought out during the development process through a joint effort of the product owner and the development team. The focus on iterative development, frequent feedback, delivery of functionality to achieve business value and close communication with the development team are in themselves merely means of identifying progress, with change in scope as a likely result.
Approach 2: Explain the nature of software development and how change is unavoidable
Most customers (and software developers) believe that software development is fundamentally a predictable process. If you would spend enough time nailing down the details upfront, no change will occur during the process as all scope and their technical implementations are known. Think about it. Only if software development is predictable, can you actually accurately predict the budget, schedule and scope of what will be delivered.
For example, one customer recently commented that if we can’t estimate correctly, we are apparently not good enough at our job. He said this in a room with four highly skilled developers, two senior analysts and one business developer that all tried to make the case that estimating accurately is impossible in software land, based on their experience.
Software development as a predictable process is a powerful misconception that makes it difficult for customers to embrace Agile software development methodologies. Let me make it clear that you can’t blame the customer for this. We need to explain how we work and where the complexities lie. We also need to stop pretending (or even believing) that it’s a predictable process ourselves.
Developing software is a highly complex process, involving many variables, unknowns and complexities. Although some parts of it are predictable, most are not because of a number of reasons:
- Different interpretations and miscommunication: Software is an intangible product. Because of this, there will be many different interpretations of what is needed and why. Worse, there are many implicit assumptions that are rarely explicitated. This makes developing software a far more complex process than building or renovating a house;
- Understanding the customer's domain: The processes that are being automated by software are intangible. To successfully automate a process, one must understand it's rules and conditions first. Software requires black-and-white decision trees. Customers don't think about their processes in this manner. And they don't have to, because processes can be executed heuristically in the 'real' world. But when automating a process, every exception, every condition, every possible path has to be taken into consideration. If a process would be fully known up front by all involved, predicting scope, budget and schedule would be a lot easier. But this is obviously almost never the case. And even in that case, the understanding of a process often changes during the development process ;
- Interdependancies: Software usually requires many different components to succesfully interact. Each component consists of hundreds if not thousands of lines of code. Each line of code can introduce new possible paths within the codebase. No human is capable of grasping the blossoming tree of possible paths that can be executed and their consequences. This makes writing code a very delicate process;
- Difficulties in communication: Developers and customers usually have a hard time understanding each other. Developers tend to think more in terms of code and technology, while customers think in terms of their process. Customers usually have a hard time understanding the technical finesses of developing software, while developers have a hard time understanding business processes. Although Agile methodologies try to limit this miscommunication by putting everyone together frequently, it is still a challenge that adds noise to the process;
- Expertise: Developing software requires many different expertises and practices. Each expertise adds a layer of complexities;
- Many unknowns: Software development entails a lot of unknowns. When you start, you often have a vague idea about what needs to be implemented, but understand that more analysis will be required at the time. These unknowns add uncertainty. Sometimes, I hear customers or development teams say they know 95% of the functionality. This does not mean that you don’t know 5% (‘oh, there’s not a lot of uncertainty’). You don’t know what you don’t know, so you can’t put percentages on it;
I think the major misconception here is that software development is often equated to writing code. This makes comparisons with assembly line work, building or renovating a house or some other physical job understandable. Although writing code is a very delicate, very hard and difficult job in itself, the majority of uncertainty is introduced not by writing the code, but by determining what needs to be written, exactly.
Maybe we should stop calling it ‘software development’ altogether, because it implies that we are merely ‘building’ something. The majority of work is often not even in ‘building’ (or writing) the code, but the process that goes on before writing any code. In a sense, calling it ‘process automation’ is probably more useful and makes customers more receptive to the creative and innovative aspects of the process (that often cause the most change).
Advice 3: Just do it, show by example
Although the above might help to explain to a customer why an Agile approach is the best approach to software development, it might not convince them as first-hand experience will do. Just doing some sprints with a customer as the product owner will hopefully show them where the complexities of software development really lie, and that change is always going to happen.
Back to the customer, and how to sell Agile
So back to the customer and the renovation analogy. It is obviously a false analogy. The challenge in software development does not lie primarily in writing the code (although that is challenging as well), but in figuring out what to write. When you renovate a house, you already have a good idea on what you need to do. As such, the scope is much smaller and less likely to move. The reality of a lot of construction work is that it still costs more as anticipated though. If a customer compares software development to renovating a house, they clearly don’t understand what it is that we are doing in the IT sector. But you can’t blame them. The analogy is easy to make, and is also made by those working in the field. I hope this blog showed you that it is more useful to focus on the advantages of the Agile approach and that customers often need to be educated on why software development is not a predictable process. If you can achieve this, it still doesn’t mean you’re going to get a signature for your project. But it will make the customer more receptive to an Agile approach.
In reality, the challenge is still to compete with the many competitors that do offer a fixed price for the same project. And customers still have to explain their budget to upper management. Although not perfect, there are several ways to do this:
- Convince them anyways: Try to convince your customer anyways and sell sprints rather than a fixed price. If the customer prioritizes correctly, the first sprint can already deliver working functionality. Also, the actual risk is only 1 sprint;
- The first sprint is no-cure/no-pay: Many customers have trouble believing in the benefits that Agile may have for them. Agile requires mutual trust. The customer has to trust that the development team will work as hard as they can, while the team has to trust that the customer honours their limits to how much they can complete in a sprint. In order to build this trust, it might help to give a customer a 'taster' of Agile. Make the first sprint no-cure/no-pay. If the first sprint was successful enough for the customer to warrant more sprints, the first sprint will be charged retrospectively. Of course, this requires fairly short sprints (one or two weeks) and a sprint backlog that really delivers value quickly.
- Fixed price for product backlog: You can also set up a product backlog with the product owner and estimate that with the development team. You can use the estimates to determine the number of sprints required, based on an estimated sprint velocity. The sprints can be translated to a budget. There are a lot of ‘estimates’ here, so use a good margin multiplier (40%-60%) of error. Be open about this to the customer and explain that you have to cover the risk of changes in scope. Also, keep in mind that point estimates for product backlog shouldn’t really be used this way. They are for relative complexity, not as a substitute for hours. Once the sprint is underway, the product owner can change the product backlog, but only be replacing items with similarly sized (in terms of points) items. Priority can of course be changed. The advantage of this approach is that it allows iterative development with a fixed price and can accommodate some change during the project. The downside is that it uses a lot of estimates that are very rough, and thus risky. It also allows for a lot of ‘contract negotiation’ during the project, as the product owner and the team argue over interpretations of vague product backlog items.
Selling your Agile projects will be incredibly hard. This blog might lead you to conclude that we know pretty damn well what we are doing at my company. But despite the internal success, we are struggling with this as well. But we are experimenting, because we really, really believe that an Agile approach is the only way to deliver high-quality software in a process that is sensitive to the customer’s needs. And that’s where the key lies; you have to really believe it yourself first. Only then can you convince you customer how they needs are met.
Good luck experimenting! And if you have any thoughts on the subject, please drop a line! I appreciate any advice.