Adaptive Agile Effort Management

Prelude:

Most business see software endeavors as projects. Resources need to be set aside for projects at defined times. And timing of deliverables often tied to marketing, budgeting, and external collaboration initiatives. The day when developers will become perfect technically, know everything about the business, be trusted by business ,and be given the freedom to do whatever they feel best has not come, and might never come any time soon. So, visibility and predictability are still required throughout a project to fit in existing business practices.

Most projects out there aren't quantum physics or sci-fi creativity work. After completing 20-40% of the stories, we should pretty much know enough for more accurate estimate. This is especially the case when proper spiking and critical path analysis are done early on. Cone of uncertainty diminishes with progress most of the time. So, re-estimation should provide more accurate adjustment.


Effort Re-estimation:

In the course of a project, there can be order of magnitude differences between original effort estimates and actual efforts spent. This points to the need of re-estimation. As Mike Cohn put it in his latest edition of "Agile Estimation and Planning (2006) : page 61-76", as long as the relative sizing of effort estimates among stories/tasks is preserved, this should be a safe thing to do. Certainly, we should re-estimate in such scenarios to maintain more accurate predictability and manageability of progress.

The question then is when and how often we should re-estimate. Should re-estimation only happen when some stories/tasks are order of magnitude out of whack? Or, should we re-estimate every iteration so that we won't forget about fine tuning our stale estimates? Should all unfinished stories be re-estimated?

Obviously, for a large project with many stories, many developers and QAs of different skill levels, ROI of such re-estimation should be considered. To make re-estimation even easier and more effective (i.e. not planning too much details too far ahead), David Leigh-Fellowes, an experienced Agile practitioner, suggested re-estimating stories in the next 1.5 iterations during IPM. The 0.5 iteration look ahead is a great idea since external dependencies can be sorted out earlier. This is especially benefitial in big organizations where collaboration is often distributed across departments and countries. Moreover, environments/code base can be shared. He prefers re-estimation by tasking out certain interesting stories in details over using actual efforts on similar stories completed in past iterations. Since re-estimation by tasking out is based on actual experience in the project then, this is actually superior to merely taking actual story points from the past and apply by analogy. He also thinks re-estimation by the whole team will make more sense in a big project setting since different developers/QAs can work on a story which has a similar counterpart with actual effort tracked, and hence different performance.

Does it mean that actual story effort points are not useful? Well, not quite since re-estimation by tasking out by the whole team is labor intensive, and can only be practically applied for 1.5 iterations look ahead, stories with wrong estimates beyond 1.5 iterations will still potentially skew overall progress projection significantly. So, a happy balance will be applying both re-estimation by tasking out for the next 1.5 iterations based on actual experience gained, and re-estimation based on actual efforts on similar stories completed for stories beyond 1.5 iterations. Needless to say that relative sizing of all the stories should be maintained in the process. This way, we achieve more accurate prediction for both short term and long term work without spending too much effort.


Effort Estimation Units:

Two well known figures in the agile community, namely Kent Beck and James Shore have reverted to the use of ideal developer days as story point measuring unit. James thinks that making estimating units more abstract makes estimating and planning more confusing and difficult. Two common opposing views are: (a) one's ideal day estimate is not necessarily the same as another's [ideal day individuality], and (b) people can be hold responsible to ideal days or be judged by whether they finish within the ideal days since it is much easier to track [ideal day fixation/accusation]. Let's dive into these opposing views one by one.

The same story implemented by different developers can take different amount of time since they have different skill sets, skill levels, experiences and approaches. This problem is actually independent of whether abstract story point or ideal developer day is used. It is a matter of "how" the estimate is done. As per the previous section on effort re-estimation, ideal day individuality should no longer be an issue because for the next 1.5 iteration, re-estimation is done by the "whole" team during, say, iteration planning meeting, and that should take into account the differences in speed between the old and new implementers. For the rest of the iterations, story effort revision from actual efforts on similar stories should still provide better estimate than not revising the story at all. After all, in an agile setting, pairing, rotation or pairs, and close communication should reduce the effort variation. And with sufficient time/scale, and mixing of stories/tasks, the aggregate estimate should still be accurate due to averaging.

If estimation is done in ideal days, it can be tempting for management or co-workers to accuse a developer's inability to finish in the estimated time frame. Or, a developer can be judged to be incompetent by not being able to finish in time. This is not desirable because estimate is still estimate, and there are often factors not fully comprehended at the time of estimation. The only thing that will come out of this is defensive behavior, damaged morale, and ineffectiveness from the commotion. Doing estimate in abstract story points might seem like a solution, however, in reality, we are shifting the problem from ideal day to velocity fixation/accusation. Developers can still be accused of not maintaining or improving on velocity. The culprit of this is not the process, but wrong mentality. Understanding the nature of the degree of unpredictability of software development is really the key to solve this. So, the good solution is not to cloud the issue, but to make it more concrete, more measurable, and hence more visible. If we do re-estimation as per described previously on a pre-defined regular basis, and repeatedly make it very clear that estimated efforts are not final, and will undergo further revisions, then we have effectively reduced the fixation on estimates. At the same time, we will have a reference point (not measurement) of performance. Variance in performance can then be managed for improvement.

Either approach requires proper communication with the clients. And periodic reminders will be needed. Yes, even ideal days can be confusing because a client might be confused about ideal versus elapsed, and whether ideal day is for one man or one pair.


Conclusion:

From the discussion above, re-estimating is a good practice for providing better prediction and progress management by adapting on better knowledge gained. By only re-estimating the next 1.5 iterations with the whole team, and then re-estimate the rest by applying actual efforts on similar stories while preserving relative sizing of the rest of the stories also reduce the problem associated with ideal day individuality. Moreover, with such standard practice and proper communication, we can also take care of the ideal day accusation/fixation issue, and making the team more productive in a sustainable manner.