An honest letter to our customers; sorry about all those lies

Tags: Agile Scrum

Dear customer of a software development agency,

With Christmas upon us, we're writing you this letter to inform you that we've lied to you. In fact, we've been lying to you for a long, long time now. We've often lied to you when you asked us if we could build a complete website or an application for a specific budget and on a given deadline. Although we gave you the  assurances you wanted to hear, we knew - in our heart of hearts - that we probably couldn't make due on them. And we didn't, a lot of the times. But we considered this a problem of the future. Something to solve down the road, by asking you for more money, more time or to accept a website or application that wasn't really finished. At that point, there was no going back for you anways. We're sorry about putting you in that position. We're sorry we haven't been honest with you. We're sorry that you're losing your faith in us.

'Now', you might say, 'I will just go to another company. They will give me the assurances I seek'. They might. But it's more likely that they will tell you the same lies that we have told you. Intentionally or not. Even though they will guarantee to deliver on time and on budget, several things are likely to happen down the road. First, it's likely that you'll be facing a very inflexible and unsympathetic account manager. Every necessary change you propose will be greeted with responses like 'Ok, but that's outside scope so you'll have to pay for it' or 'Only when the entire system is done and there's budget left'. It will feel like they don't want to understand how important these changes are to your business. Second, chances are that the same account manager will call you -  somewhere during the project - to inform you that they need more money and more time. Third; although your application might be delivered on time and on budget, chances are that it will not work as stable, user-friendly and slick as promised. After all, if budget and deadline have to be met, quality suffers.  So despite all those initial assurances, you still have to pay more and/or wait longer or accept an unfinished product.

The reality is that many, many IT projects fail completely, don't deliver the promised quality or don't deliver on time and/or on budget. Despite all those initial assurances. How did it get this far? Well. There are several reasons.

The most important one is that software development is just not easy. Despite what you might think (and what we sometimes tell you when you casually inform if something 'is hard to build'), it's not just a matter of writing code. You see, software development is about solving a business problem. That makes it an inherently innovative, creative process. Because the complete solution is not initially known, we will have to experiment.  Despite our best intentions we can't possibly predict in great detail and with great accuracy what we're going to build upfront. Ideas will pop up, our understanding will grow, feedback will create new insights, the problem will change, change will happen, even though we all agree not to let that happen. It will. Because that's the nature of the innovative, creative process that software development really is.

Another reason is that we really want to be there for you, our customer. If you press us with questions like 'how much is it going to cost and when is it done?' we feel obligated to provide you with an answer. After all, we don't want to lose you. We also understand that you prefer solid assurances instead of best intentions, even though we can't really provide them. So we lie, intentionally or not. Because we can't predict in detail the effort involved, we guess and multiply those guesses, threefold, with Pi or some other random value. That makes the estimates sound more convincing. After that, it's mostly a matter of luck. You should know about this, dear customer. You can't possibly be happy with false guarantees that are backed by fictitous and heavily inflated estimates that you end up paying for. We are not happy with it either.

The matter of the fact is that we are keeping each other hostage in a process that does not work. Because you desire assurances that we really can't give, we have devised a very complicated approach that's designed to give us a false sense of security. We start by having a lot of lengthy meetings about what we're going to be building. And then some more. We document everything in hefty, complicated, technical documents that supposedly completely describe what we're going to be making. Because this is your one moment to influence the end result and the budget, you will feel a strong urge to pile on every possible idea and functionality you can think off. Even though it's unlikely anyone will ever use it. Better to have it in there anyways. Although we all have very different interpretations of said documents, we all sign off on it. And then we estimate the effort it will take using the pragmatic 'guess * pi' approach. We're both victims here, really. You end up paying a lot more than you could have if we would have started with the functionality that you really need. But you also end up paying a lot of more if luck turns in our favor and we somehow manage to deliver on time. We end up paying if we can't deliver on our promises. Either way, we both suffer financially depending on who's lucky. And so do the users of the system we're working on.

So, where do we go from here? We should begin by not longer pretending that software development is easy and predictable. It is clearly and empirically not. It is an innovative, creative process, even for small projects. The nature of such a process is that we don't sufficiently know what the end result is going to be and how we are going to get there, exactly. Because we don't know this, we can't give you long-term guarantees on delivery dates and budgets. Instead, let's work together closely to create your website or application step by step and evaluate very frequently if we're still on track. Let's begin with those parts that are most important for your business. In short iterations, preferably only a few weeks, we will work to deliver (in order of priority) those pieces of functionality that are most valuable to your business. And you pay for that, and that alone.

"So I basically just give you money and don't get any assurances?!", you may exclaim. No. Although we can't guarantee to deliver the entire system on budget and on date, we can guarantee to deliver working software every iteration. Because iterations are short, the complexity is limited and assurances are easier to give. After every iteration, you can reconsider if you have faith in us and in the project and want to continue with another iteration. The advantage for you is that you can trust that you are paying only for working functionality, and not for ficticious, heavily inflated (guessed) estimates. We can also incorporate new ideas, suggestions from users and changes in future sprints, without our nagging account-manager breathing in your neck. This process is more transparant, more honest and more realistic. All that energy that is normally lost on slaping each other around with specification documents and re-negotiations about budgets and deadlines can now be spent on actually building working software. And quality will not suffer. It's really better for all of us. This is what Scrum and Agile are all about.

Dear and valued customer. We hope that you are willing to embark on this journey with us. Many have done so before you, and are now more satisified with the software that is being developed for them. Although we can't give them all the guarantees they might seek, they do get working, high-quality software delivered in short iterations. That must be worth something to you as well,

We hope to hear from you, soon,

Software developers

Christiaan Verwijs
Christiaan Verwijs

Scrum Master, Trainer, Developer & founder of Agilistic