Improving dist-upgrades reliablity and user experience

A lot of our users do full dist-upgrades rather than reinstalling for every single release.

It's a good thing that we support it and a very important feature, unfortunately it's a rather fragile feature, especially when doing LTS-to-LTS upgrades.

This session is here to discuss ways to allow for faster, more reliable dist-upgrades.
Some ideas:
 - Split download from upgrade (because the download can take quite some time)
   - need to lock it so that new packages are not installed at all or do it in a separate location
   - download without changing sources.list at all
   - need to make sure that people don't reboot if sources.list has changed
     - Just have apt inhibit shutdown? what if something goes wrong, fallback?
       It's not going to stop someone from pressing the power switch
 - Run upgrade in a minimal failsafe environment
 - Use snapshotting features, overlayfs, ... when available to "test" the upgrade
 - Detect known issues prior to upgrade and warn the user about them

== misc ==
 * make it easier, and improve documentation, to use the release-upgrader for people with internal mirrors to override internal mirror warning
 * is there a command line way to switch between: lts-only or all releases? (apart from editing files under /etc/update-manager/release-upgrades)

Beginning of a design spec: https://wiki.ubuntu.com/ReleaseUpgrades -- mpt

Work items:
[stgraber] Get the dist-upgrader to download all the packages into a temporary location (with its own sources.list) to avoid locking apt during that time: TODO
[stgraber] Look at depending on metacity (but be aware of flavours) so we have a stable window manager to run the upgrade: TODO
[stgraber] Implement a two step upgrade in update-manager, first run resolves and downloads, second run resolves, downloads anything left and upgrades in a minimal desktop environment: TODO
[brian-murray] look into conffile prompt detection beforehand using the code in unattended-upgrades: POSTPONED
[mvo] document how to upgrade with local mirrors and with preserving with third party repors and how to set NonInteractive mode by default on upgrade: TODO
[mvo] support upgrades from a totally isolated system with only internal network and mirrors: TODO
[mvo] phased upgrades! like phased updates using a percent number in the meta-release file: TODO
[stgraber] In case of system reboot during upgrade, try to run the text upgrader at boot time, if it fails, try to fix dpkg/apt and try again: TODO
[brian-murray] Add tests for the two phase upgrade where "interesting" things happen during the two (adding repositories, installing packages, ...): POSTPONED