When does Waterfall make more sense than Agile/Scrum?

Thursday, June 02, 2011 » Agile, Project Management, Software Development

A couple of days ago I was asked the question "When does a Waterfall development approach make more sense than Agile/Scrum?"  My first reaction was "never," because based upon my experiences in utilizing Agile/Scrum, it can be applied to nearly any development project.  Upon a few seconds of reflection upon the precise words of the question, however, I decided to provide a more detailed answer.  Waterfall certainly makes more sense to many people in situations where the early sprints of an Agile/Scrum project would not yield any runnable product.   The key is that it "makes more sense."

The reasons that waterfall "makes more sense" to many during the early phases of a project are that:

  1. Milestones in waterfall are often placed on the master project plan to reach out to point where there are tangible deliverables.  Short Agile/Scrum sprints may not provide sufficient time to produce tangible deliverables.
  2. In waterfall, progress between milestones is often not measured or measured as percentage progress.  This is believed to be more conducive to greenfield development up and until the point that runnable product exists.
  3. There is a common misconception that Agile/Scrum backlogs are strictly feature driven and early-stage development may not expose actual features until considerable work goes into foundational development.

The key to using Agile/Scrum in early stage development lies within definitions for the process, a strong understanding of the process and benefits, as well as the flexibility to accept some substitute notions for deliverables.  Indeed it is these substitute notions that make the least sense to some people and cause them to believe that waterfall is more appropriate for early-stage greenfield development.  

In an Agile/Scrum framework, the product backlog does indeed normally reflect a desired feature set.  Through conversations I have found that there is a common misconception among those who do not practice Agile/Scrum that the process of preparing for a sprint consists of directly transferring prioritized features from the product backlog to the sprint backlog, accounting for time.  In practice, a key part of the process involves decomposing feature implementation into tasks with associated time requirements.  Tasks  are designed to be manageable and measurable chunks of work in the 4-16 hour range.  Hence, the sprint backlog doesn't contain features, it contains tasks, the completion of a sprints-worth may or may not result in the complete implementation of a feature.  The implementation of Tasks as a "measurable" (rather than a "deliverable") provides a concrete and meaningful way to measure progress in a project that has not yet yielded (and presumably will not yet yield for some time) demonstrable product.

I've heard the suggestion made that both the sprint backlog and product backlog should be maintained in separate staging tables, with the sprint backlog being cleared and reset for each sprint.  I have found it more effective to return partially completed features to the product backlog with a notation about estimated incompletion percentage.  This allows the flexibility to re-prioritize product requirements for each sprint, and, if necessary, re-decompose the feature into a new set of tasks based upon previous development experiences.    By keeping the product backlog stateless, agility is maintained - one of the key benefits of Agile/Scrum over a waterfall process.  The product's feature set priorities can be changed practically on the fly (well, each sprint anyways) to match company, customer and/or marketplace priorities and requirements.

Many texts on Agile/Scrum place undue emphasis on being able to demo a shippable product at the end of each sprint.  During early-stage greenfield development, this simply isn't possible.  Suspending this practice until there is a demonstrable product is, IMHO, a better option than losing the fine-grained control and agility offered by an Agile/Scrum approach.

One of the best benefits that comes from a properly executed Agile/Scrum approach is that, in my experiences, Agile/Scrum teams remain highly motivated by being able to measure progress frequently, and by being able to accept new tasks without becoming buried in them for entire projects.  I've heard the reverse arguments that waterfall team-members develop a stronger sense of "ownership" for their particular long-term areas, but have noted this to be a double-edged sword.  "Owners" are notoriously poor at  cross-pollinating knowledge, and as a result losing an "owner" for any reason is usually a major set-back for a project.  "Owners" also frequently "fall into ruts," burning out or becoming complacent in their areas.  

comments powered by Disqus