As software developers, we live challenging lives! Especially in an Agile environment, where we have to bring all our skills to bear to finish the sprint on time and deliver business value for the Product Owner. Over the past few months, I've been working with an Agile Team in two-week sprints improving an existing and quite complicated planning environment that my company has been developing over the past few years. Being part of this team and taking part (again) in actual development has been a blast, but it has also re-emphasized for me how important Agile Software Development principles really are.
In this blog I would to like to rehash some of those principles, so that others may avoid the same mistakes. I will borrow greatly from the wealth of literature surrounding the topic, so if you're interested in more details just check out the references below the post.
Why Agile development is hard and difficult work
Agile Teams work in very different environments than more traditional (Waterfall) teams. Although they both provide their fair share of challenges, Agile Teams often face a greater challenges:
- High pressure: First and foremost, there is more pressure on Agile Teams. They have to continuously deliver working code that generates value for the Product Owner. Deadlines are short and the team is not shielded from customers by layers of analists and account managers;
- Rotting code: Because of the pressure, many Agile Teams tend to jump into coding and skip planning and design. Although it might make the team appear fast, it will cause **the codebase to become progressively less transparant and progressively harder to maintain. Teams that accumulate this kind of technical debt will see a decline in their velocity and an increase in technical issues and bugs;
- Mistakes are costly: Because speed and efficiency are important, the risk for mistakes is also greater. A team can't spend half a sprint experimenting with a new technology and throw it away halfway through the sprint if it doesn't work out. Teams have to develop strategies for experimenting in such a manner that the success of the sprint does not depend on it;
So, being part of an Agile Team is hard work. Thankfully, a lot of developers have come and gone before us and we can learn from their experiences. In this blog, I will summarize some of the principles that I consider most useful within Agile Teams (in no particular order):
1. Just in Time Design & coding
In a sprint, you don't have a lot of time to implement the functionality described in the Sprint's Backlog. So you have to be efficient and economic with your time. This is a good thing, because it helps maintain focus. Most developers I know love their job and they love writing good code. But they often get carried away writing really, really clever code that is not actually necessary. Here are some pointers to keep your coding and design JIT:
- Don't use design patterns for the sake of design patterns: Good developers love writing crafty, intelligent code. There's nothing wrong with that. But a lot of that code ends up being over-engineered. Design patterns are intended to increase maintainability at the cost of added complexity and (somewhat) reduced transparancy. Applying design patterns is very useful, but only when you have proof that you are going to need it within the sprint or very soon after. So don't apply Template Methods or Strategy Patterns when you don't expect more strategies to be used in the very near future (this or the next sprint). Don't use Singletons when threading is not an issue yet. And most importantly, don't use Inheritance for classes that are not really related (but share some logic). Apply a 'We'll cross the bridge when we get to it' mentality and write only what you need. If you follow the other principles described in this article and the many, many related books and articles (see the references below) your code will be easy to extend when the time comes;
- Don't optimize when you have no proof that it's necessary: Good developers think ahead. They anticipate problems and performance issues that may arise. This can lead to heavily optimized designs and code. Caching may be used to keep often-used data in memory, batching can be used to avoid hitting the database too often and multithreading can be employed to improve algorithmical performance. I'll be the first to defend the value of optimization, but optimizing your code too early is very costly. It will obviously take time to implement and reduces the transparancy of code and design. But worse, a lot of optimization is not necessary because the anticipated heavy load never actually occurs or it was done in places where the reduced performance does not bother the end user. I'm a big fan of using a more optimistic and less defensive approach. Assume that performance is ok until proven otherwise. Make sure that the Product Owner is involved in deciding where optimization is necessary, to avoid spending a lot of time optimizing code that is rarely used or not a problem to end users anyways;
- Don't do 'Big Upfront Designs': Although this is more common in waterfall projects, some Agile Teams go through the motions of setting up a functional and technical design every sprint (with class diagrams, activities schemas and the such). Although this is useful for very complicated problem domains, most 'Big Upfront Designs' just waste the creative energy of a team and limit Agility. For every team it will be different, but I prefer to draw and scribble some stuff on a whiteboard with the team to create a shared understanding of the problem and decide on the best approach. We then split up and write code to make it work (pair-programming is useful here) and check on each other's progress regularly to change course when necessary. If it doesn't work, try other strategies quickly. This has worked very well for many, many sprints and has not compromised the quality of the code. This does not mean that you should not do any design altogether, just do it when it's necessary;
2. Think, write, test, refactor
Example from the field: For a planning application, I rewrote part of the scheduling logic for a planning application. The possible number of scenarios was huge, including a lot of very exotic ones. Instead of spending time on a design to deal with all the scenarios, I wrote unit tests and code to deal with the more basic scenarios first. This took half the sprint. The act of writing tests and code improved my understanding to the point where I realized that a more abstract approach could solve all the scenarios. Something I missed before, even though I spent quite a bit of time thinking about it (on the way home, under the shower, you know the drill). I took three days to rewrite the existing code and add support for the exotic scenarios and got it to pass all the tests;
Developers are clever people. They like solving complicated scenarios and turning them into coherent, logical, clean algorithsm that they can be proud of. It's also a bit of an ego thing, of course. Most developers are perfectionists. This can be a tricky thing. Perfectionism can result in several sprint-blocking problems:
- Developer-block: I've experienced this a few times. A developer block is the result of wanting to write perfect code from the get-go, but not being able to produce anything that lives up to that level. Sometimes it's better to just write some crappy code that works and work from there instead of waiting on a 'eureka' moment that isn't going to happen and refactor it afterwards into better code;
- Perfect code that is not working: It's very easy to start writing 'perfect code', but fail to deliver anything of value at the end of the sprint because time ran out;
The reality of development in a sprint is that a) things usually take more time and that b) unexpected things may pop up that you have to deal with. No matter how clever you are, your code is never going to be 100% perfect the first time. Don't try to fight this by being very (too?) clever, but apply a process of writing code that fits with this reality. I usually apply these steps, which are fairly common among Agile developers (see also Fowler, 1999):
- Think: Take some time to think the problem through, making sure you understand it as best as you can at the time. It really helps to consider as many real-life scenarios of the functionality as you can, as it gives you an idea on how to test the code. Write down test cases that either pass or fail (e.g. 'If I delete a user, he is gone when the page reloads' or 'When an employee is on vacation, he is not available to work during that period'). If you are using Test Driven Development, this will be the time when you write your unit tests;
- Write: Write the minimum amount of good code you need to get the test cases to pass.Don't optimize your code just yet. Don't go overboard on design patterns, except for those that make your code more flexible for change and don't reduce transparancy. Good code is testeable, simple, decoupled and applies Object-Oriented Programming principles;
- Test: Test your code thoroughly, either through unit- and/or regression tests or through manual testing. The act of writing and testing your code will have improved your understanding of the problem domain.
- Refactor: You should now refactor any code that you don't feel happy with, as long as it benefits the sprint. After refactoring some code, make sure your test cases still pass and write new test cases where necessary.
Now, there's one big 'but'. The above approach can result in rotting code if the team never gets the time to refactor code. If you see this happening, discuss with your team why this is happening. The velocity may be too high or the team might be understaffed. Don't go on like this, as it will bite the team in the ass eventually.
3. Unit testing (really!)
I can't remember the number of times I thought 'Wow, this is cool' when reading articles about unit testing, only to skip unit testing because it took too much time. I'm sure you'll relate. Most of the developers I know understand the theory, see the benefits but don't write unit tests. That's ignorant, stupid and dumb.
I now consider unit testing to be a necessary part of the development process and do it wherever possible. Not writing unit tests will cause problems down the road, guaranteed. So, why should you write unit tests?
- Red/green development: Unit testing is often praised for it's ability to give developers something to work towards. This is often called 'red/green coding'. At first, all tests fail because no code has been implemented. But as code is being written, tests start passing and more and more turn green. This is obviously quite fulfilling for a developer and makes them increasingly trust their code;
- Deepens understanding: The act of writing down unit tests and thinking about how to test you code will deepen your level of understanding of the problem domain greatly;
- Documents your code: Unit tests, when they are kept small (and they should!), document how your classes are to be used. Other developers may learn from this;
- Results in better code: Writing unit tests will naturally force you to write better code. Unit testing a class is easy when your class is loosely coupled to it's dependancies, when it has a clearly defined interface and when it has one responsibility;
- Aids refactoring: Unit test are pretty much a prerequisite for refactoring. If you have unit tests that cover most or all of your code, refactoring will become a lot easier because you can check if your changes broke existing code;
Writing unit tests is the best way to improve the Agility of the development process and avoid some of the pitfalls of the short iterative cycles of Scrum/Agile software development. Writing unit tests takes extra time, especially for existing systems, but the process itself is so beneficial to the overall development process that you shouldn't skip it.
I should note that I don't normally test all possible code paths. I do test my business logic and data access classes. I don't normally test my UI or database objects. Writing tests for this code is usually a lot of work, they quickly break when you change something and testing it manually is easy. The value for me is in writing tests for my business logic, which is where the most critical bugs will occur.
4. Write Object-Oriented code (OO), not procedural code
When I look back at code I wrote many years ago, I recognize how badly I understood Object Oriented Design back then. Most of my code was blatantly procedural, with some classes here and there to structure the code into logical chunks like glorified modules. But I failed to apply Object-Oriented principles in a useful manner because I didn't understand it. It took several years before I truly started to see the benefits. If there's one advice I can give any developer, it is to try very hard to understand OO as early as possible.
I do blame some of the books and articles for this. A lot of examples are too convenient and use real-life objects (like cars, dogs or airplanes) to explain that everything can be considered in terms of objects and behavior. They make a good point, but applying that knowledge is very difficult when you are dealing with very abstract business processes and business rules. A lot developers can identify obvious objects (like users, products, etc) but fail to capture processes like 'completing an electronic payment' or 'matching employees based on their preferences in a schedule' in terms of objects and behavior. The result is a lot of procedural code, which can be easily identified by a large number of methods, large classes, many switch statements, deep hierarchies of if-then-else statements and the like, making the code very hard to read and understand.
So, why is good Object-Oriented code superior (in case you are still wondering)?
* More natural: Object-Oriented code expresses processes and business rules in terms of related objects and behavior, making it easier to model your code in a language that is more natural than procedural code. This improves communication between developers and customers, although it takes some abstraction skills from all parties involved; * Transference of knowledge: Object-Oriented code improves transference of domain knowledge. When applied correctly (following principles and patterns), other developers more easily understand your code. * Inheritance: Object-Oriented code allows developers to group shared logic and expose it through a common interface throughout the application (encapsulation). Furthermore, other classes can derive from this class and extend or change that logic if it makes sense. Code re-use is minimized and maintainability maximized; * Application of design patterns: Object-Oriented code allows the application of many well-known design patterns that will improve the maintainability of your code;
So, read books on the topic and make sure you understand it. Everyone in the team should understand the relevance of OO to Agile Development (or any other kind of software development process, for that matter).
5. Apply Agile Design Patterns and Principles
There are many good books on this topic (see the references below), but applying Agile Design Patterns and Principles to your code will greatly improve maintainability. Here are the design patterns that I like the most (and are pretty much required):
- Dependancy Injection: Decouple dependancies. According to some authors the most important one, and I agree. I do see it more as a result of the principles below. Dependancy Injection is the process of decoupling your classes. One class should not depend on the concrete implementation of another to work. This sounds fairly abstract, but I think my blog post on the topic might be useful here. The biggest advantage of decoupling your classes is that the testability if your code improves massively. In fact, unit testing is just not going to work without it! Coincidentally, dependancy injection also forces you to think about the interfaces of your classes and how they are supposed to interact;
- Composition over Inheritance (Gamma et. al. 1994): Favor composition over inheritance. You're pretty much implicitly doing this when you are injecting dependancies, but composition is the process of extending and/or changing the behavior of a class by changing the classes it contains or uses, instead of using inheritance (deriving a new class from another or changing the inheritance tree). Inheritance is very cool, but it can make your code very hard to understand when your inheritance tree becomes deep (>2 levels). It also makes your code brittle. How often have you bumped your head when you had to change many classes because you changed some parent class? When you use composition, you split the behavior of a class into many small classes that you 'compose' together with (for example) dependancy injection. So, one class will use many other small classes to achieve it's goal;
- Single Responsibility Principle (Martin, 2002): One class should have a single responsibility, and it should encapsulate that completely from other classes. What exactly constitutes 'one thing' is a matter of taste, but the size of the class is a good general guide. If your class starts growing in the number of methods, properties and lines of code, it's probably accumulating responsibilities. Some classes do many things. They get data from a database, process it and save it again. Through different methods of boolean parameters, they often allow multiple strategies to be executed. This is not a big problem if your class is very small (every method is like 5-25 lines and there are very few methods). But when your class grows, maintainability becomes harder and harder. Small classes, that have one (small) responsibility are easier to understand, easier to change and - most importantly - easier to test in isolation. This is where unit tests come in. Writing unit tests usually naturally pushes developers to write small classes, because they are easier to test. This is why writing unit tests is so useful;
- The Open-Closed Principle (Martin, 1996, Meyer, 1988): Classes should be open for extension, but closed for modification. When you write a class, you should write it in such a manner that you can change the behavior of the code without rewriting the original code. You should be able to extend from the original class and override behaviors or (preferably) use composition to change the behavior of the class. The point of doing this, is that you can guarantee that the original code still works. You won't break anything. Instead, you are extending the behavior of a class through other means (inheritance, composition, overriding). This is also where many design patterns excel, like the Template Methodor Strategypatterns. It's easy to get lost in very complicated code designs when you are trying to protect (and abstract) your code against any kinde of change. Instead, Martin (2002) argues that developers should take a more optimistic approach and protect against change only when it's very obvious. If other parts of the code change in the future, the initial adaptation will be costly. But the code should be rewritten to protect against further changes of that type. I like Martin's metaphor of guns and bullets; don't put on full body armour. It takes a lot of time to put on and slows you down. Take the first bullet. It will hurt, but put in the effort to put on body armour against that type of gun and make sure you won't be hit by it again. You might be hit by other guns in the future, but don't assume you will be;
There are many other useful Agile Design principles, like the Liskov Substitution Principle, the Interface-Seggregation Principel and others. There are many good books on the market, so I suggest reading those. Check the references below.
Writing good code is very challenging. Writing good code in an Agile environment is even harder. At least, that's what it might look like. Because Agile teams have to be economic with their time, so they require strategies that allow them to maintain their flexibility throughout the sprint. Thankfully, many developers have gone before us and there's a lot we can learn from their experiences. In this blog, I summarized some of my own lessons learned with those of many other developers. Take the time to read the references, don't assume you already know or understand it all and - above all - push yourself to become a better developer. Being part of an Agile team is certainly going to be a great help :)
Have fun coding!
Martin, Robert C. (2012). Agile Software Development: Principles, Patterns and Practices;
Fowler, Martin (1999). Refactoring: Improving the Design of Existing Code;
Fowler, Martin (2003). Patterns of Enterprise Application Architecture;
Meyer, Bertrand (1988). Object Oriented Software Construction;
Martin, Robert C. (1996). The Open-Closed Principle(PDF);
Gamma, E., et. al. (1994). Design Patterns: Elements of Reusable Object-Oriented Software;
Martin, Robert C. (2008). Clean Code: A Handbook of Agile Software Craftmanship;