Notes from Agile Estimating and Planning by Mike Cohn
Estimating Project Size
"Agile teams separate estimates of size from estimates of duration. To understand the distinction, suppose I am tasked with moving a large pile of dirt from one corner of my yard to another. I could look at the pile of dirt, assess my tools (a shovel and a wheelbarrow), and directly estimate the job at five hours. In arriving at this estimate, I bypassed any estimate of size and went directly to an estimate of duration.
Suppose instead that I look at the pile and estimate its size. Based on its dimensions, I estimate the pile to contain about 300 cubic feet of dirt. This is my estimate of the size of this project. But an estimate of size alone is not useful in this case. We want to know how long it will take to move the dirt. We need to convert the estimate of size (300 cubic feet) into an estimate of duration.
A label on my wheelbarrow says it has a capacity of 6 cubic feet. Dividing 300 cubic feet by 6 cubic feet, I decide that moving the dirt will take 50 trips with the wheelbarrow. I estimate that each trip will take 3 minutes to load the wheelbarrow, 2 minutes to walk to the other side of the yard and dump the dirt, and 1 minute to walk back with the empty wheelbarrow. Total trip time will be 6 minutes. Because I anticipate making 50 trips, my estimate of duration is 300 minutes or 5 hours."
All we need to know is whether a particular story or feature is larger or smaller than another one.
Estimating Size with Project Points
Story Points are Relative
"Story points are a unit of measure for expressing the overall size of a user story, feature, or other piece of work. When we estimate with story points, we assign a point value to each item. The raw values we assign are unimportant. What matters are the relative values. A story that is assigned a two should be twice as much as a story that is assigned a one. It should also be two-thirds of a story that is estimated as three story points."
A team can begin with the smallest feature and assign it 1 point. The team can then estimate all other features relative to the smallest feature.
A team can also try to identify a medium-sized feature.
Velocity
"Velocity is a measure of a team's rate of progress. It is calculated by summing the number of story points assigned to each user story that the team completed during the iteration. If the team completes three stories each estimated at five story points, their velocity is fifteen. If the team completes two five-point stories, their velocity is ten."
"A key tenet of agile estimating and planning is that we estimate size but derive duration. Suppose all of the user stories are estimated and the sum of those estimates is 100 story points. This is the estimated size of the system. Suppose further that we know from past experience that the team's velocity has been ten points per two-week iteration, and that they will continue at the same velocity for this project. From our estimate of size and our known velocity value, we can derive a duration of ten iterations or twenty weeks. We can count forward twenty weeks on the calendar, and that becomes our schedule."
Velocity Corrects Estimation Errors
Fortunately, as a team begins making progress through the user stories of a project, their velocity becomes apparent over the first few iterations. The beauty of a points-based approach to estimating is that planning errors are self-correcting because of the application of velocity. Suppose a team estimates a project to include 200 points of work. They initially believe they will be able to complete twenty-five points per iteration, which means they will finish in eight iterations. However, once the project begins, their observed velocity is only twenty. Without re-estimating any work they will have correctly identified that the project will take ten iterations rather than eight.
SGB: It will be interesting to measure the velocity of one team of students in a given class compared to prior semesters of that same class. One reasonable measure of the effectiveness of this approach to teaching may be to evaluate the velocity of students the first time they engage in a development project and compare that to the subsequent experiences.
Estimating in Ideal Days
"...managers are able to work an average of only five minutes between interruptions (Hobbs 1987). Even if the typical developer is interrupted only one-third as often, that is still an interruption every fifteen minutes."
"Problems can arise when a manager asks a team member the inevitable question: "How long will this take?" The team member responds, "Five days," so the manager counts off five days on her calendar and marks the day with a big red X. The team member, however, really meant to say, "Five days if that's all I do, but I do a lot of other things, so probably two weeks."
A number of interruptions occur in the form of emails, meetings, etc.
"A software developer who is told to multitask loses a great deal of efficiency while switching between two (or more) tasks."
SGB: This is going to significantly reduce productivity for student teams. They are simply interrupted too frequently.
When calculating ideal days, a team should do so in the following context:
- Pretend the story being estimated is the only thing you will work on.
- Everything you need will be on hand when you start.
- There will be no interruptions.
One Estimate, Not Many
If you choose to estimate in ideal days, assign one aggregate estimate to each user story. Some teams are tempted to estimate a number of ideal days for each individual or group who will work on a story. For example, such a team might estimate that a particular user story will take two ideal days from a programmer, one ideal day from a database engineer, one ideal day from a user interaction designer, and two ideal days from a tester. I've seen teams then write the estimate on the story card with a different-colored marker for each role or on a different-colored sticky piece of paper for each role that is affixed to the story card.
In the vast majority of cases, my advice is not to do this. This level of focus on the individual roles on a team shifts team thinking away from the "we're all in this together" mentality we'd like to exist on an agile team. Further, it vastly increases the amount of work necessary to plan a release. If each story is assigned an estimate for each role that will work on the story, the release plan should realistically take each role into account. This means we'd have to track velocity and remaining work for each role as well.
Techniques for Estimating
Estimates are Shared
"Estimates are not created by a single individual on the team. Agile teams do not rely on a single expert to estimate. Despite well-known evidence that estimates prepared by those who will do the work are better than estimates prepared by anyone else (Lederer and Prasad 1992), estimates are best derived collaboratively by the team, which includes those who will do the work. There are two reasons for this.
First, on an agile project we tend not to know specifically who will perform a given task. Yes, we may all suspect that the team's database guru will be the one to do the complex stored procedure task that has been identified. However, there's no guarantee that this will be the case. She may be busy when the time comes, and someone else will work on it. So because anyone may work on anything, it is important that everyone have input into the estimate.
Second, even though we may expect the database guru to do the work, others may have something to say about her estimate. Suppose that the team's database guru, Kristy, estimates a particular user story as three ideal days. Someone else on the project may not know enough to program the feature himself, but he may know enough to say, "Kristy, you're nuts; the last time you worked on a feature like that, it took a lot longer. I think you're forgetting how hard it was last time." At that point Kristy may offer a good explanation of why it's different this time. However, more often than not in my experience, she will acknowledge that she was indeed underestimating the feature."
The Estimation Scale
Studies have shown that we are best at estimating things that fall within one order of magnitude (Miranda 2001; Saaty 1996).
Because we are best within a single order of magnitude, we would like to have most of our estimates in such a range.
Two estimation scales I've had good success with are
1, 2, 3, 5, and 8
1, 2, 4, and 8
Each of these numbers should be thought of as a bucket into which items of the appropriate size are poured. Rather than thinking of work as water being poured into the buckets, think of the work as sand. If you are estimating using 1, 2, 3, 5, and 8, and have a story that you think is just the slightest bit bigger than the other five-point stories you've estimated, it would be OK to put it into the five-point bucket. A story you think is a 7, however, clearly would not fit in the five-point bucket.
Deriving an Estimate
Expert Opinion: "In an expert opinion-based approach to estimating, an expert is asked how long something will take or how big it will be. The expert relies on her intuition or gut feel and provides an estimate."
Analogy: "When estimating by analogy, the estimator compares the story being estimated with one or more other stories."
Disaggregation: "Disaggregation refers to splitting a story or feature into smaller, easier-to-estimate pieces."
The author recommends using planning poker. Planning poker combines each of the above estimation techniques.
Participants include all developers on the team.
"At the start of planning poker, each estimator is given a deck of cards. Each card has written on it one of the valid estimates. Each estimator may, for example, be given a deck of cards that reads 0, 1, 2, 3, 5, 8, 13, 20, 40, and 100. The cards should be prepared prior to the planning poker meeting, and the numbers should be large enough to see across a table.
For each user story or theme to be estimated, a moderator reads the description. ... The product owner answers any questions that the estimators have. However, everyone is asked to remain aware of the effort/accuracy curve (Figure 6.1). The goal in planning poker is not to derive an estimate that will withstand all future scrutiny. Rather, the goal is ... a valuable estimate can be arrived at cheaply.
After all questions are answered, each estimator privately selects a card representing his or her estimate. Cards are not shown until each estimator has made a selection. At that time, all cards are simultaneously turned over and shown so that all participants can see each estimate.
It is very likely at this point that the estimates will differ significantly. This is actually good news. If estimates differ, the high and low estimators explain their estimates. It's important that this does not come across as attacking those estimators. Instead, you want to learn what they were thinking about.
The group can discuss the story and their estimates for a few more minutes. The moderator can take any notes she thinks will be helpful when this story is being programmed and tested. After the discussion, each estimator re-estimates by selecting a card. Cards are once again kept private until everyone has estimated, at which point they are turned over at the same time.
In many cases, the estimates will already converge by the second round."
The Right Amount of Discussion
"Some amount of preliminary design discussion is necessary and appropriate when estimating. However, spending too much time on design discussions sends a team too far up the effort/accuracy curve of Figure 6.1. Here's an effective way to encourage some amount of discussion but make sure that it doesn't go on too long.
Buy a two-minute sand timer, and place it in the middle of the table where planning poker is being played. Anyone in the meeting can turn the timer over at any time. When the sand runs out (in two minutes), the next round of cards is played. If agreement isn't reached, the discussion can continue. But someone can immediately turn the timer over, again limiting the discussion to two minutes. The timer rarely needs to be turned over more than twice. Over time this helps teams learn to estimate more rapidly."
When to Play Planning Poker
Teams need to decide when to play planning poker. Probably at the beginning of a project and then for the first couple of iterations. After that, the team may be comfortable with its estimates.
Why Planning Poker Works
"Now that I've described planning poker, it's worth spending a moment on some of the reasons why it works so well.
First, planning poker brings together multiple expert opinions to do the estimating. Because these experts form a cross-functional team from all disciplines on a software project, they are better suited to the estimation task than anyone else. After completing a thorough review of the literature on software estimation, Jørgensen (2004) concluded that "the people most competent in solving the task should estimate it."
Second, a lively dialogue ensues during planning poker, and estimators are called upon by their peers to justify their estimates. This has been found to improve the accuracy of the estimate, especially on items with large amounts of uncertainty (Hagafors and Brehmer 1983). Being asked to justify estimates has also been shown to result in estimates that better compensate for missing information (Brenner et al. 1996). This is important on an agile project because the user stories being estimated are often intentionally vague.
Third, studies have shown that averaging individual estimates leads to better results (Hoest and Wohlin 1998) as do group discussions of estimates (Jørgensen and Moløkken 2002). Group discussion is the basis of planning poker, and those discussions lead to an averaging of sorts of the individual estimates.
Finally, planning poker works because it's fun."