Staging ISOs before daily smoke testing
Trying to improve ISO testing by running some automated testing before image publication.
Blueprint information
- Status:
- Not started
- Approver:
- Pete Graner
- Priority:
- High
- Drafter:
- Gema Gomez
- Direction:
- Needs approval
- Assignee:
- Canonical Ubuntu QA Team
- Definition:
- New
- Series goal:
- Accepted for raring
- Implementation:
-
Unknown
- Milestone target:
- None
- Started by
- Completed by
Whiteboard
During the quantal cycle there have been times where the daily ISO images had bug or problems on them, which hit many users and or automated jenkins testing.
Ideally we want to prevent users from downloading borked images and hitting/reporting bugs over and over again.
We should discuss and possibly implement something to help with this, e.g.:
* consider uploading images to a staging area, running jenkins test and if jenkin tests pass then publish the iso images
* or for example push jenkin test failure notes to the cdimage / iso-tracker (like the oversize warning)
* think about potentially yanking borked images from cdimage.u.c
* communicate "why there was no daily build yet?", e.g. by pushing "FTBFS" notification to iso-tracker & cdimage.u.c
* communicate respin process in a more automatic way, e.g. on the iso tracker / cdimage.u.c
Objective:
- Try to run some automated testing before the images are published
- Work with Unity testing team to make sure unity works (hook unity jenkins to be dowstream from the desktop-default jenkins test)
Work Items
Work items:
[cjwatson] Pre-publish images to a staging area, and work with gema on how to trigger testing: DONE
[cjwatson] Pull jenkins results on above smoke-test and then publish (or not) based on results: INPROGRESS
[cjwatson] Keep the image that doesn't pass $somewhere, so we can debug what broke: DONE
[pwlars] Discuss potential triggering methods with Colin for the initial publish/no-publish call: DONE
[pwlars] Get firewall rules changed to allow ssh to cdimage from a QA system to request the images get copied from pending->current: DONE
[pwlars] Modify image sync scripts to sync from pending instead of current: DONE
[pwlars] Setup a new parameterized job to handle requesting the pending->current transition when smoke tests pass, fix job hierarchy to trigger this at the proper place: DONE
[pwlars] modify smoke job creation scripts to include the new pieces to call mark-current, use them to create the new test jobs for saucy: DONE
[pwlars] fix download-
