Fortunately, the metrics gathered during each Agile release cycle can be very useful for that purpose. Once we gain a good understanding on what is being worked on, for how long and by how many people, we should be able to extrapolate this to forecast future releases.
Let’s take a look at what activities take place during a typical release cycle:
- Plan release
- Write tests
- Write code
- Fix bugs
Repeat ad nauseam until done.
In this context, the tasks that programmer take on can be categorized as follow:
This is the body of work identified during the planning stage. This is the raison d’être of the release. For the most part, this covers new features or so called non-functional requirements such as performance improvements. This is what we’ll want to talk about when the product gets released. It’s a common planning trap to think that this is the only work that will take place.
As the name implies, this is work that was unforeseen at the beginning of the release but is required to be completed before the product can ship. It can be further refined as follow:
Change in Requirements
This should not be unexpected. In fact, any Agile methodology assumes that there will be changes down the road. This is not a problem per se as long as there is a mechanism to trade features, extend duration or increase resources.
Omissions in Planning
Because the planning period is relatively short, it is assumed that not everything is fully specified or researched upfront. As development proceeds, new pieces are discovered and introduced into the plan.
Defects can be considered a side effect of software development. They can affect the product in different ways:
- Regression - these are defects introduced when working on new code. They usually impact unrelated functionality that used to work before.
- Defect in new code - these are problems in a newly coded feature. The feature does not work quite as expected. It’s not done.
- Newly discovered existing bug - these are bugs present in the previous version of the software. They are either known or newly discovered during the course of the release and prioritized to be fixed now because they affect existing users.
Using The Past to Project In The Future
A believable long term plan needs to layout new features on a timeframe against a set of resources. Because there is only a limited amount of time and people allocated to thinking through the issues upfront, the initial estimates are always very rough.
It is then very difficult to predict how much time certain things will take or how many people will be required. To increase accuracy, you’d have to invest more time and resources, which is not practical, mostly because these are the very same resources that are critical to deliver against the current release plan.
“Prediction is very difficult, especially about the future.”Niels Bohr (maybe)
You need a model to help forecast based on imperfect data. You could decide to simply pad all the current estimates, but you’d still need to figure out a factor that’s sufficient.
A better model is to find a way to anticipate the additional work that will be generated by the introduction of a brand new feature. With that model, we can layout a plan that should allow sufficient room in every release to accommodate for all the work, planned, unplanned and bugs.
Let’s take a look at historical data from a real software project:
|Planned Features [pts]||198||300||145||100||160||153||86|
|Actual Completed [pts]||696||946||527||419||432||694||303|
|Actual Completed Features [pts]||290||304||183||151||214||284||145|
|Planned/Completed Features ratio||146%||101%||126%||151%||134%||186%||169%|
|Actual duration [days]||39||55||56||50||50||55||30|
The first row in the table contains the budgeted points for each release. That number represents the sum of all planned work as determined at the end of the planning phase for that release. The release schedule was set based on that number and the historical team velocity.
The second row represents the total actual points completed during the release, including features and bugs. This represents that amount of effort to get the release out.
The third row represents the actual completed feature points only. Next row computes ratio of planned vs. actual work.
It’s very tempting to compute a velocity based on total completed points and use it as a predictor for how much work the team can accomplish. While it’s true that it is a measure of total effort, it can’t be used against planned work only.
What we want to determine is that given 1 point of planned feature work, how much unplanned work and bugs will be generated and thus how long will it take to complete. Then normalize this for team size and you get a factor that can be used for long term planning.
To get there, I’ve computed a velocity for each release based on planned work divided by actual release duration (in days) and number of engineers. I then use the median value across release as a factor to turn estimated planned points into calendar days. With that in hand, I created a simple model where scope, duration and resources are linked. Set two values and the third one gets computed automatically. This model becomes very handy to run through what-if scenario during roadmap sessions. It provides answers to business questions such as:
- How long would it take to do feature X?
- Can you meet deadline Y?
- How many more engineers would it take to do Z in Y time frame?
1. In this context, releases refer to the more traditional software delivery model vs. a hosted model. Enterprise software, desktop application or mobile apps all have some sort of release overhead that make continuous delivery challenging.