Agile Story Points - Scope and Effort Management

Abstract: Using story points to manage both scope and effort is convenient but not ideal. Story points should be split for scope and effort management. Such separation will ease project analysis, planning, and communication. Features or business values should be the reference for scope story points. Efforts should be tracked separately. Having scope story points separately tracked, better product and financial management of the project can be achieved. By tracking effort story points independently, re-estimation for better progress management can be done without impacting scope measures. Appropriate periodic re-estimation of effort story points empirically basing on actual efforts instead of stale and somewhat arbitrarily assigned values at the beginning improves predictability of delivery schedule and collaboration coordination. Because of planned re-estimation, there should be no fear of using ideal work days as measures, which reduces confusion and difficulties in planning and estimating. Visualization of the correlation between the scope and effort story point velocities provides another means to assist with work prioritization.

1. Introduction

In one of the Agile practices, business value delivering functionalities/features are organized into units called stories. These stories are often relatively sized and recorded using a parameter called story point. With consistent and proper story sizing, story points are good indicators of scope and effort. Total story points are often interpreted as the total scope as well as the total effort for the project. However, scope and effort are very different concepts and have different quantitative and qualitative variations in the course of a project.

As more knowledge is gained about the nature of the stories, previous effort estimations often need revision. In other words, story points will need to be changed to reflect the new effort estimation. As a result, scope measure will inevitably and inappropriately change as well since both effort and scope are tracked with story points. Depending on whether the new total story point is higher or lower than before, scope inflation or deflation will occur respectively. By having an effort story point attached to each story instead, we can safely do re-estimation without causing inaccurate scope change. With re-estimation as part of the process, the fear and worry of estimating in ideal man days also disappear (see http://agilesanity.blogspot.com for details )

On the other hand, a highly valuable and complicated feature/functionality might take very little effort to complete. For example, the shopping cart of a web application can be easily implemented by purchasing and integrating existing tool kits. On the contrary, a low value feature/functionality might take a lot of effort to deliver. For instance, to add contextual help for an air ticket booking web application might sound like a simple one liner feature, but it actually entails a lot of work behind the scene. The gist of these examples illustrate that feature size and effort size are not necessarily positively correlated with business value.

Tracking scope story points separately has the additional benefit of increase ease in associating business values to the stories. This will help identification of project progress variance analogous to the use of planned/earned/actual values analysis common place in established project management discipline.

The correlation between scope and effort velocities is also another good visualization tool for understanding project progress, resource usage, and hence prioritization.

But the most immediate benefit of scope and effort story point differentiation is reduction of abstraction. Clarity is gold.

However, if we make story point management too complicated, then the ROI might not be justifiable. As in anything, balance is the key. There is a Confucius saying, "The fools never worry about balance, but the smarts try too hard to balance and losing it." So being agile in spirit is the true way.

2. Scope Management with Scope Story Points

Scope story points, or simply scope points, can be allocated as simple as one point per identified story. So, if we have 19 stories, then we will have 19 scope points, which represent the total scope at the moment. This is acceptable when stories are supplemented with effort story points.

However, there are more useful ways of assigning scope points. For example, we can associate scope points with business values. Let’s say the project is about producing a PC car racing game, which is expected to make a net profit of 5 million dollars in 3 years. Then, the upper bound for total scope points can be assigned 5 million (the lower bound can be the cost of the project). In other words, completing 5 million scope points will mean that all essential features of the game are developed. With this lump sum figure as the reference point, consideration of addition or removal of features/functionalities can be made by taking into account whether such action will increase enough sales/profit to justify the extra cost based on effort story points (to be discussed later). So, better decisions can be made towards scope changes, and resource allocation.

A few more words on scope point assignment with the PC car racing game example. Sometimes, there are critical inter-dependent stories which aggregate completion is required for a product to be shippable. If the majority of stories are under this category, then the burn up or burn down chart of scope points will be useful. However, if, say, less than half the stories are critical and inter-dependent, then, these charts will not be as useful because the scope velocity of non-critical independent tasks does not depict a true picture of how far the project is progressing towards completion if the critical path is lagging behind. This is the case because the risk and unknown are not explored in the critical path. The simplest way is to track critical inter-dependent stories separately.

With scope point come scope velocity, and scope burn up chart. The former is not as useful as effort velocity (to be addressed later) in terms of completion prediction. The latter, on the other hand, is the better indicator of earned values than effort burn up. For more information on earned value management, please see "Project Management Body of Knowledge - 3rd Edition".

3. Effort Management with Effort Story Points

See "Adaptive Agile Effort Management" post in http://agilesanity.blogspot.com for details.

4. Relationship between Scope and Effort Story Points

The correlation between scope and effort story points is a good indicator of ROI management. This helps prioritizing of stories. For example, if the burn up charts of both effort and scope indicate positive correlation, then we are delivering values proportional to effort. If the scope burn up is at a higher rate than effort, then it means we are delivering higher values with relatively lower cost. The contrary will indicate that we are spending a lot of efforts delivering little values. Of course, there are times we have to do this because of resource constraints or inter-dependencies among tasks which constitute the stories. And even among stories, dependencies can be hard to avoid at times.

5. Conclusion

In this article, the cost and benefits of separating story points into scope and effort points are explained. It is not difficult to separate them. The simplest way is to assign one point per business value delivering story, and then track effort points just like we do with story points nowadays. More sophistication management of scope points is to associate business values. There are many ways to do this. How much efforts should be spent in sophisticated scope point management is subject to the ROI of such endeavors.

Adaptive Agile Effort Management

Prelude:

Most business see software endeavors as projects. Resources need to be set aside for projects at defined times. And timing of deliverables often tied to marketing, budgeting, and external collaboration initiatives. The day when developers will become perfect technically, know everything about the business, be trusted by business ,and be given the freedom to do whatever they feel best has not come, and might never come any time soon. So, visibility and predictability are still required throughout a project to fit in existing business practices.

Most projects out there aren't quantum physics or sci-fi creativity work. After completing 20-40% of the stories, we should pretty much know enough for more accurate estimate. This is especially the case when proper spiking and critical path analysis are done early on. Cone of uncertainty diminishes with progress most of the time. So, re-estimation should provide more accurate adjustment.


Effort Re-estimation:

In the course of a project, there can be order of magnitude differences between original effort estimates and actual efforts spent. This points to the need of re-estimation. As Mike Cohn put it in his latest edition of "Agile Estimation and Planning (2006) : page 61-76", as long as the relative sizing of effort estimates among stories/tasks is preserved, this should be a safe thing to do. Certainly, we should re-estimate in such scenarios to maintain more accurate predictability and manageability of progress.

The question then is when and how often we should re-estimate. Should re-estimation only happen when some stories/tasks are order of magnitude out of whack? Or, should we re-estimate every iteration so that we won't forget about fine tuning our stale estimates? Should all unfinished stories be re-estimated?

Obviously, for a large project with many stories, many developers and QAs of different skill levels, ROI of such re-estimation should be considered. To make re-estimation even easier and more effective (i.e. not planning too much details too far ahead), David Leigh-Fellowes, an experienced Agile practitioner, suggested re-estimating stories in the next 1.5 iterations during IPM. The 0.5 iteration look ahead is a great idea since external dependencies can be sorted out earlier. This is especially benefitial in big organizations where collaboration is often distributed across departments and countries. Moreover, environments/code base can be shared. He prefers re-estimation by tasking out certain interesting stories in details over using actual efforts on similar stories completed in past iterations. Since re-estimation by tasking out is based on actual experience in the project then, this is actually superior to merely taking actual story points from the past and apply by analogy. He also thinks re-estimation by the whole team will make more sense in a big project setting since different developers/QAs can work on a story which has a similar counterpart with actual effort tracked, and hence different performance.

Does it mean that actual story effort points are not useful? Well, not quite since re-estimation by tasking out by the whole team is labor intensive, and can only be practically applied for 1.5 iterations look ahead, stories with wrong estimates beyond 1.5 iterations will still potentially skew overall progress projection significantly. So, a happy balance will be applying both re-estimation by tasking out for the next 1.5 iterations based on actual experience gained, and re-estimation based on actual efforts on similar stories completed for stories beyond 1.5 iterations. Needless to say that relative sizing of all the stories should be maintained in the process. This way, we achieve more accurate prediction for both short term and long term work without spending too much effort.


Effort Estimation Units:

Two well known figures in the agile community, namely Kent Beck and James Shore have reverted to the use of ideal developer days as story point measuring unit. James thinks that making estimating units more abstract makes estimating and planning more confusing and difficult. Two common opposing views are: (a) one's ideal day estimate is not necessarily the same as another's [ideal day individuality], and (b) people can be hold responsible to ideal days or be judged by whether they finish within the ideal days since it is much easier to track [ideal day fixation/accusation]. Let's dive into these opposing views one by one.

The same story implemented by different developers can take different amount of time since they have different skill sets, skill levels, experiences and approaches. This problem is actually independent of whether abstract story point or ideal developer day is used. It is a matter of "how" the estimate is done. As per the previous section on effort re-estimation, ideal day individuality should no longer be an issue because for the next 1.5 iteration, re-estimation is done by the "whole" team during, say, iteration planning meeting, and that should take into account the differences in speed between the old and new implementers. For the rest of the iterations, story effort revision from actual efforts on similar stories should still provide better estimate than not revising the story at all. After all, in an agile setting, pairing, rotation or pairs, and close communication should reduce the effort variation. And with sufficient time/scale, and mixing of stories/tasks, the aggregate estimate should still be accurate due to averaging.

If estimation is done in ideal days, it can be tempting for management or co-workers to accuse a developer's inability to finish in the estimated time frame. Or, a developer can be judged to be incompetent by not being able to finish in time. This is not desirable because estimate is still estimate, and there are often factors not fully comprehended at the time of estimation. The only thing that will come out of this is defensive behavior, damaged morale, and ineffectiveness from the commotion. Doing estimate in abstract story points might seem like a solution, however, in reality, we are shifting the problem from ideal day to velocity fixation/accusation. Developers can still be accused of not maintaining or improving on velocity. The culprit of this is not the process, but wrong mentality. Understanding the nature of the degree of unpredictability of software development is really the key to solve this. So, the good solution is not to cloud the issue, but to make it more concrete, more measurable, and hence more visible. If we do re-estimation as per described previously on a pre-defined regular basis, and repeatedly make it very clear that estimated efforts are not final, and will undergo further revisions, then we have effectively reduced the fixation on estimates. At the same time, we will have a reference point (not measurement) of performance. Variance in performance can then be managed for improvement.

Either approach requires proper communication with the clients. And periodic reminders will be needed. Yes, even ideal days can be confusing because a client might be confused about ideal versus elapsed, and whether ideal day is for one man or one pair.


Conclusion:

From the discussion above, re-estimating is a good practice for providing better prediction and progress management by adapting on better knowledge gained. By only re-estimating the next 1.5 iterations with the whole team, and then re-estimate the rest by applying actual efforts on similar stories while preserving relative sizing of the rest of the stories also reduce the problem associated with ideal day individuality. Moreover, with such standard practice and proper communication, we can also take care of the ideal day accusation/fixation issue, and making the team more productive in a sustainable manner.