Dr. Dobb's Digest July 2009
In 2007, the company I work for purchased the rights to a medical practice management and billing system from a third-party software development firm. Technically, the software is an n-tiered product written in .NET that can be fully hosted at a client site or on our servers. A project was launched to rebrand the product, interface the product into existing business lines, test and debug the product, and add market differentiators.
Because the product was new, the work was highly exploratory with rapidly evolving requirements that could only come to light by learning the product. We decided to use Agile development supported by Scrum and elements of Extreme Programming. The entire team went through a tremendous learning curve in both learning Agile development and the product itself. Without Agile's benefit of small iterations and the chance to adjust our progress, the project most likely would have had significant difficulties. After 18 months, the project reached a high level of agile maturity, even incorporating numerous lean principles.
Our Scrum process was solid: Two-week sprints with a demo at the end, seven people consistently on the sprint teams, burndown charts, sprint planning for each sprint with whole team commitment, estimation by story points, velocity measures, and a story point estimation session with the entire team one week before each sprint.
At first, the business side of the company was not accustomed to the responsiveness of Agile development. The norm was a formal waterfall approach to software development that could require several quarters to develop a software release. But it didn't take long for the business to develop such a hunger for change that requirements (user stories) were barely stable enough for a two-week sprint. It was not unusual for the product owner group to give top priority to stories that were not well thought out, forcing development to cobble together a sprint full of unknowns. Changes also worked their way directly into the sprints -- in mid-sprint new stories were swapped with stories that hadn't been started. It seemed as though a two-week sprint was too long as the business side of the project swung from rigid change control to demanding immediate responsiveness. Something had to be done.
The management team brainstormed around ways to make Scrum as flexible as possible on intake. We considered ways to shorten the sprint length, the estimation sessions, sprint planning, and sprint demo length even further, all with the goal of keeping people working on stories that were of highest value to our product owner and increasing productivity. (See the sidebar for ideas on Scrum.)
|
Where Can Scrum Be Leaned Out?
Our thought process was to take lean principles from Mary Poppendieck's Implementing Lean Software Development: From Concept to Cash and apply them to the meta processes of Scrum, posing questions such as:
—C.S. |
Our conclusion was that the basics of software Kanban, as described by Corey Ladas, would fit the bill.
While Kanban means "visual card" or "board", using Kanban in software development is much more than a card display board. It involves reducing waste and emphasizing the ability to change, partly through limiting availability of inventory (the story cards of Kanban). In our process, the work is divided into a series of stories. The stories then end up on a series of lists: the Most Wanted list, the Pre-ready list, the Kanban board (Ready, In Process, and Done). The only status the Kanban team saw were the Ready, In Process, and Done lists. Figure 1 shows the story progression.
The product owner group creates rough cut stories and sequences them in a Most Wanted list. Analysts groom the top stories in the list and, once they are detailed, put them in the Pre-ready list in a tool called "Jira." The scrum master moves stories from the Pre-ready list to the Ready list on the Kanban board only when the development team pulls a story from the Ready list. The number of stories in the Ready list is kept small -- four to eight -- for two reasons: only the highest priority stories are worked, and stories are flexible until the Kanban team pulls the story. We also limit work by maintaining a strict rule that a teamlet (one or two people) can only work on one story at a time unless a story is blocked; then, and only then, can a second story be pulled in.
While the analysis work may seem like a mini-waterfall, it is certainly not. The analysts learn the details of the story and collect information for the development group. When a story is pulled into the Kanban, the analyst is a first-class member of the teamlet, just like on a sprint team.
When a teamlet picks a story, the story is estimated in ideal hours. If the story proves to be more than a week's worth of work, we meet with the analysts and break the story into smaller segments, if at all possible.
As stories are completed, the teamlet demonstrates them to the analysts and product owner group. A short retrospective follows with the teamlet, very much like Scrum, but at a story level rather than sprint level.
Figure 2 is our Kanban board. The In-process list is further subdivided into Started (stories that are being analyzed and digested by the team), Code/Test (stories that are being coded via test driven development), Awaiting Test (stories that are awaiting acceptance testing), and Pending Release (stories that are acceptance tested but are awaiting a patch build). Each story is written on a green card with an orange tag indicating patch number, and a blue tag indicating who is working on the story.
Every day we have standup meetings where the commitments for the day are written on the purple tags next to the stories. There is also an issue list at the bottom of the board for technical debt or other items that need to be addressed by either the teams or the scrum master. An typical story will progress through each list on the Kanban board in just a few days.
As you can see in Figure 2, acceptance testing and patch builds are currently batched. This is due to staffing restrictions and other circumstances beyond our control. While this situation is not perfect, our Kanban board makes the batching under Awaiting Test and Pending Release highly visible.
Table 1 shows a number of points where our Scrum practices were modified to work with Kanban. We continue to use a number of Scrum practices where it adds value; such as daily standups, story cards, and information radiators.
Kanban has a number of clear advantages. In particular, there is a large degree of flexibility in story prioritization. As stories are not cast in stone until they are pulled into the Kanban, there is very little waste developing stories that aren't played and significant flexibility in reprioritizing the backlog. Stories are typically a day's worth of work, so emergency fixes can be pulled in almost immediately without disrupting the work in progress. Stories are autonomous units of work so there is no need to group stories based on a sprint goal, or release goal for that matter -- there is less emphasis (and time spent) on planning sessions.
Kanban is highly productive. The amount of work completed in just a few months rivaled that of six months of Scrum because there is little wasted time. A number of things contribute to the productivity. I found that team members were not distracted with the end of the sprint push like they were in Scrum. Because high-priority stories can be pulled from the backlog without waiting until the end of a two-week sprint, there is less temptation for management to change priorities in the middle of work, resulting in reduced context switching for the developers. The developers have something they can produce quickly, leading to a good sense of accomplishment and a chance to react quickly to lessons learned from a story. Altogether, the team and the business are pleased with the application of Kanban.
However, Kanban does not come without a number of challenges. With teamlets forming and reforming on a story-by-story basis, there is a danger of team cohesion being lost. Fortunately, we did not experience cohesion problems because most team members have had together for a long time. Kanban simply became a new way to pull work. Several Scrum measurements are lost. Predictability by sprint is lost (no sprint) and velocity is no longer measurable. The only metric is cycle time though the Kanban board -- which only works if stories are all the same size. Also, it is hard to tell when a certain story in the backlog will be completed because new stories can shuffle the priority frequently.
Kanban seems like a logical next evolutionary step to Scrum, but only in the right circumstances -- projects where the work doesn't need to be completed in groups of stories. Kanban has a number of advantages such allowing extensive flexibility of the backlog, more flexibility in staffing, and the ability to manage work in the face of uncertainty.
Kanban is successful in our environment, although it should be applied carefully. Kanban has increased productivity and given the business a chance to react quickly to lessons learned, and, because of our team experience, cohesion has not been a problem.
I can see difficulties with using Kanban in a product development project; a pure Scrum- based approach may be better. The benefits of Scrum, such as iteration 0, sprint planning, release planning, and team protection are lost with Kanban. Collaborative product design and technical design are part of the sprint planning, which is not part of Kanban. Even the teamlet concept, rather than Scrum's whole team effort, can lead to a certain level of lost collaborative design. If you decide to use Kanban for product development, you must have strong leadership from a product owner.
Kanban can be effective, I know from experience, but it should not be applied without understanding the tradeoffs. Keep up with the literature on the Internet; while it's a new idea, there is a wealth of information already available.