Shrink the image build pipeline
When preparing milestone releases, we often need to turn around fixes quickly. The pipeline from a developer source upload to a full set of updated image builds on all architectures is currently somewhere in the region of nine hours. We would like to make this much quicker.
Blueprint information
- Status:
- Started
- Approver:
- Steve Langasek
- Priority:
- High
- Drafter:
- Colin Watson
- Direction:
- Approved
- Assignee:
- Colin Watson
- Definition:
- Approved
- Series goal:
- Accepted for precise
- Implementation:
-
Good progress
- Milestone target:
-
ubuntu-12.04-beta-1
- Started by
- Colin Watson
- Completed by
Whiteboard
Work items:
[adconrad] Spread legacy livefs building infrastructure across more machines as a stop-gap: DONE
[adconrad] Migrate live filesystem building into Launchpad, so that it can be scheduled across a larger number of machines, which would particularly help ARM: POSTPONED
[adconrad] Write launchpad-buildd slave implementation for live-build jobs: POSTPONED
[adconrad] Write buildd-manager and LP API implementation for live-build jobs: POSTPONED
[adconrad] Test live-build/
[adconrad] Rollout live-build/
[cjwatson] Reorganise germinate to expose a useful Python interface: DONE
[cjwatson] Convert Launchpad to use python-germinate, aiming to reduce cron.germinate's run-time from around 10 minutes to around 2 minutes: DONE
[cjwatson] Move germinate before apt-ftparchive in Launchpad: POSTPONED
[cjwatson] If possible, make the publisher run every 30 minutes rather than every hour: DONE
[cjwatson] Fix the CD image system's archive mirror to be less inefficient in its handling of symlinks, which should save in the region of 10 minutes per serialised image build: DONE
[cjwatson] Consolidate main/universe archive mirrors on the CD image system into one, reducing races between certain sets of parallel builds: DONE
[nick-moffitt] Move CD image building to the new SAN: DONE
[lamont] Switch powerpc live filesystem builds to a faster machine (RT#48569 filed): TODO
[cjwatson] Restore some timestamp logging in live filesystem builds (Debian #630089): DONE
[cjwatson] Switch cdimage mirroring to mirror directly from cocoplum: DONE
[cjwatson] Look at transfer speed between nusakan and cocoplum and make sure sane (measured at ~330Mbit/s over rsync, seems good enough to me): DONE
[cjwatson] At least vaguely start on the giant debian-cd merge: TODO
[adconrad] Write a master CD build script (after live-build/
Work Items
Dependency tree
* Blueprints in grey have been implemented.
