This is a repost of a series of article I originally published for Songbird.

Wrestling Bugzilla into Shape

Because so much of our existing workflow occurred in Bugzilla, we’ve decided to use it as a central database to drive our process. Every actionable project artifact lives in Bugzilla, a Feature, Story, Task or Bug. From a release standpoint, the only actionable items are Story, Tasks or Bug and hence, we only track these.

Release Train

Every release train has a name and is used to create a target milestone in Bugzilla. This allows us to put items in release buckets.

Priority Field

As we create stories, tasks and bugs we assign them to the release milestone and place them in a priority bucket. Each priority level indicates how the item is being treated during the release:

• P2 - must have for the release, next candidates for promotion to P1
• P3 - nice to have for the release, we may ship without it
• P4 - nice to have and low risk. This represent the kind of work that we probably will undertake during the QA phase as it is low risk and won’t likely cause a regression.
• P1 has a special meaning. It is used as a flag to represent what is being committed by the team for the current weekly iteration. Everything in P1 signals our tool that this is our “plan” for the week and is under active development (We are considering changing this in the future to use the assigned bug functionality since Bugzilla 3.2 will now allow you to accept and assign a bug in one operation).

From an overall release tracking standpoint, the tool includes P1, P2 and P3. P4 are not considered part of the plan. If we get to them during QA, they are freebies.

Costing

We would not be able to track anything without costing. Every item is being assigned a cost of 1, 2 or 3 based on complexity, scope and past experience. We also use 0 when an artifact exists but is not actionable by itself. For instance, we often start creating stories and cost them. Then, as we refine our plan, we may create one or several tasks to represent the particular engineering implementation of that story. When this occurs, each tasks has a cost, and the story cost field is zeroed out so we don’t double count. Also, the story to task relationship is being identified with the blocks/depend on field of Bugzilla.

Weekly Iterations

Every Monday, we plan our work for the week. Every member of the team elects items to P1 status. We then tally up the total points and determine if the plan is sound. Once we are all in agreement, the plan gets locked. This simply means that the tool takes a snapshot of all the items marked as P1 and saves their ids.

Daily Reports

The main function of the bot is to compute delta and generate a daily report. Let’s take a look at every sections of the report.

Iteration

A simple number to represent the iteration (one week)

Plan

This section represent the plan of records as snapshot at the beginning of each iteration. Once the plan is locked, it becomes immutable.

Duration

Duration of the iteration. Usually 5 working days, but it automatically adjusts for holidays.

Remaining

Total points for P1, P2 and P3 items which status is open (e.g. UNCONFIRMED, NEW, ASSIGNED or REOPENED)

Planned

Total points for P1 items that are opened.

Planned Velocity

Normalized points per work day.

Actual

This section represent the actual metrics computed at the end of each iteration. The iteration ends on Sunday at midnight.

Intake

Represent everything that was either added or removed to the P1 list (the plan) during the iteration.

Intake Velocity

Normalized intake points per work day.

Completed

Total point of everything that was marked fixed during that iteration.

Team Velocity

Normalized completed points per work day.

Carried over

Everything planned at the beginning or added during the week and not completed by the end of the iteration.

Environment

Originally, this was introduce to track distractions outside of the core release that could impact the team velocity. The idea being that it would give us a gauge of what the “environment” was like during that period. This columns would capture everything open or closed in the special ASAP Bugzilla milestone. We used this classification to track urgent issues, infrastructure work, etc. As our process matured, the need for this diminished and we no longer use this.

Here is a sample of such report for the Fugazi release.

What to Look For

Every day an email report is being generated allowing us to track progress to date. For every column described above, a link is provided. The link points back to Bugzilla and contains a list of ids for the items included in the list. Every item can then be reviewed independently.

At our weekly planning meeting, we usually start by reviewing what was carried over from last week. Ideally, we want this to be a null set, but the reality is that it often contains a fair amount of items. It’s primarily due to intake during the iteration. Items are being added because of regression introduced and discovered, new tasks being identified or slight adjustment to the plan. This is definitely an area we need to improve on.

The next view is the planned items for the new iteration. This allows us to make some tweak to our plan before we commit.

The intake views are very useful to analyze what is being added or removed during the cycle.

The team velocity gives us a measure on how well we are doing and how much load we think we can carry during the next iteration.

Finally, the remaining points tally gives us our burn down number. This is how much work is left and by dividing this with our historical velocity gives us a sense of how long it will be until we are done.

Open Source Library

SDPBOT is built as a Ruby library. The source code has been open sourced under the MIT license. It is available at http://code.google.com/p/sdpbot.

We have made no attempt to make it generic, so do expect to do some work to customize it for your own needs. If you’re using Bugzilla and want to generate some type of reports you will hopefully find the library to scrape and instantiate objects from Bugzilla very valuable. Enjoy!