Further improvements to the Checkbox release process

We have made good progress in the last six months on delivering well tested and stable releases of Checkbox to our PPAs. The aim of this blueprint is to continue down that road and reduce the amount of time this process takes so that we can do even more frequent releases while remaining rock solid.

Right now the release process for Checkbox involves quite a lot of manual tasks:

1.) Preparing the release for testing by creating a release branch for both checkbox and checkbox-certification and building it in a testing PPA
2.) Performing a full run of the client-cert.whitelist (using checkbox-certification-client), sending results to the certification site and analysing the results for any unusual failures.
3.) Checking results of automated checkbox-certification-server testing on the certification site
4.) Performing functional tests described in https://docs.google.com/a/canonical.com/spreadsheet/ccc?key=0ApQ2JshzVOLydHhDZU8zQ3JZNjgtZ0FyTGVnYS1rS0E&usp=sharing
5.) Conducting release meeting with representatives from different parties using Checkbox
6.) Preparing release notes consisting of changelog and whitelist diffs
7.) Copying binaries from testing PPA to the public PPA
8.) Preparing the Ubuntu Checkbox release

All in all these tasks account for at least one full person day. In this cycle we want to attempt to automate and otherwise improve those tasks that satisfy the equation of bringing maximum benefit for minimum effort.

These have been identified as:

1.) Creating files containing release notes (mostly) automatically
2.) Automatically preparing the upload of Checkbox to the development version of Ubuntu. Note that for this there may be some precedent in other teams so we should try and reuse their work. See http://blog.didrocks.fr/post/Unity%3A-release-early%2C-release-often%E2%80%A6-release-daily
3.) Improving the already existing automated testing of checkbox-certification-server to send a notification on failure, where 'failure' is defined as any test failing or the submission to the certification site failing.

We may also like to take time to discuss a strategy for versioning Checkbox 0.16 such that we do not run into the problems encountered in the last development cycle regarding the frozen Ubuntu package superseding the 'stable' version.

We should also look into a way to manage packaging in such a way that we don't end up with frequent conflicts in debian/changelog caused when there is a delay between proposal of a merge and the merge itself.