Although many thought leaders and researchers have doomed so called waterfall model, it still goes strong. I have worked with many companies, and most of them apply waterfall model in some form. I won’t discuss about the reasons for applying waterfall model in this blog post, but I point out one important problem related to waterfall approach, which is the problem with late integration and test.

Imagine a 1-2 years long project with huge up-front specification phase, loooong development phase, and finally a busy integration and system test phase. All this is planned on MS Project chart, requirements are frozen, hundreds of pages long specs have been written and signed off long time ago, and so on. This is a classic project managed with waterfall model.

Management and project leader have agreed that development team creates a system defined in the specs, and all stakeholders are waiting for the delivery. People change their thoughts of things by sending email. Sometimes they talk about the project in front of coffee machine, but most of the discussion is done in emails. Time goes on, and one day project manager realizes that he/she is in trouble. Time has come. System must be integrated and tested before delivery.

“Houston, we have a problem“.

Why late integration and test can cause a big trouble?

Once a colleague told me “in my next project I will make sure ALL requirements are found, and after that I will drop them one by one to a huge MS project plan in order to make sure nothing goes wrong and nothing gets forgotten in my project“. I replied, “you know what? It won’t work. You won’t succeed. Just welcome change and use adaptive planning“. He didn’t understand, and kept on repeating his idea..

Shortly said, the problems have been there but nobody has noticed them. Integration activities finally force the team to face the problems that should have been fixed long time before. For example during integration huge amounts of interface problems show up, and much refactoring and recoding is needed. There are fundamental misunderstandings, wrong assumptions, communication gaps, blaming and a chaos. Project runs in crisis mode and steering group takes the lead. What’s the problem?

The problem is that applying waterfall model to software project, which is always new product development, does not work. Software projects are inventive, and because of that predictive planning is not possible. In big software projects things change, and therefore creating big up-front specs and plans fail. Such predictive planning doesn’t welcome change, and it is rather unrealistic and risky.

When predictive planning and waterfall model is used, integration and test is usually planned in the end of the project. Software developers need feedback from testing in order to clarify requirements and develop code that fulfills the requirements. If integration and testing is done after months of coding, it is obvious that problems show up, and usually there is no time to fix the problems. For some reason waterfall based plans typically include delivery right after system tests.

How the problem could be fixed?

Iterative and agile processes apply adaptive and continuous planning. After initial planning and staging phase team starts 2-4 weeks long sprints that include continuous integration and tests. In fact XP process requires daily builds and automated regression tests, which is very much different from waterfall approach.

By combining agile or iterative process and risk-driven project planning we can pretty well avoid the problem of late integration and test.

How do you see this? Do you different experiences or thoughts about the problem of late integration and test?

Photo: certified_su