Thursday, February 19, 2009

Fostering (friendly) inter-team rivalry by build monitoring

The team I'm currently managing is split into three distinct groups working on things we refer to by the names Platform, Corgi and Javascript. Given that my startup is currently in stealth mode that's about all I can tell you about what we are doing.

It'll come as no surprise to regular readers (since I started Electric Cloud that we are using a continuous build system (current it's hudson) and to keep everyone informed there's a prominently positioned flat screen display showing build status.

Here's a screenshot (each team has chosen a dog breed to use as a codename for their part of the code):

To build this page a simple Perl script reads information directly from Hudson via its API converting Hudson JSON objects into Perl structures.

Initially the builds were simply presented in the order given by Hudson, and a simple color scheme was put in place: red means the build is broken, yellow means the tests are failing and green means everything is ok.

When builds are broken the Perl script pulls the list of culprits from a Hudson API and names and shames the people who broke the build by showing their photographs against the red background.

But the team came up with the idea of also using the order (from top to bottom) as a way of indicating just how good or bad build stability is. After much arguing about the best algorithm (and a number of prototypes which themselves caused one of the teams to accelerate unbreaking their build) we settled on the following:

When a build is broken or unstable it appears at the bottom of the list, and the longer it's been in that state (i.e. the longer it is since the last green build) the further it is down the list. This is pretty easy to determine since Hudson has a lastStableBuild API for each build.

The green builds are ordered by a computed value called 'Health'. The script gets the status of all the builds (for a particular component) within the last week and computes the percentage of builds that were green. The higher the health the higher on the build monitor screen.

Doing so creates some friendly rivalry. No one wants a really broken or unstable build, but even within the Green Zone the teams are competing to keep their code in good shape all the time.

And by having the build monitor accessible as a web page it can be viewed on the build monitor flat panel display, from any web browser, or from a mobile device like the iPhone:

Now I can stay home and watch what the team is up to, and harass them about broken builds before I've had my breakfast.

No comments: