system images for servers

* explicitly not on the table to disable apt, this will always be an option
* what kind of image could we build for a server system image?
 * not going to cover all use cases; initially, a small image similar to cloud images that we can run workloads on top of
 * sabdfl suggestion to call it the Ubuntu Core image
 * cjwatson says we should be able to produce something quite similar to the current ubuntu core tarballs - but not identical... click, system-image, bootloader e.g.

 * bootloader challenges: can we make this support both UEFI and BIOS?
  * compatibility with older hardware that people are likely to test on, plus VMs, implies BIOS
  * probably install both bootloaders, as the binary bits (grub-pc-bin, grub-efi-amd64-bin, etc.) don't conflict; may need some care with grub-install calls

* on the phone, we reboot to recovery to apply updates.
 * for server, we may want to avoid this
 * the system-image spec does include a 'bootme' flag; for server we might apply the updates live (mounting the rootfs rw only in a namespace, so the rest of the system processes can't see it) and reboot only if required and in any case only after upgrade
 * s-i "reboot" step (really, apply-update) is hookable, so possibly write a separate app for applying updates and then have s-i call this thing. might need small amount of dev around separating application of update from reboot (i.e. you'd still do that on server if a bootme flag exists, but only after apply-update) - probably also need D-Bus API changes.

* should system-image metadata include an indicator to restart apps, as opposed to restarting the whole system?
 * barry suggests that this metadata should be in the app instead
* do we care about distinguishing between service restart and system restart?
 * yes, because the BIOS itself is quite slow on many of these
 * but maybe we could do kexec, CRIU
  * not for the first iteration

* want to support juju local provider as an interface for this?
* install lxc and use it an lxc host

* need to support installing services as click packages
 * this means click packages need to be able to attach to a system hook that lets them provide upstart jobs or systemd units
 * cjwatson proposes this being systemd from the start
 * there is consensus on this
 * systemd units provided by the click package, but should be filtered
  * also needs extended, not just filtered
 * systemd namespacing support allows unsharing, but not mapping

* which services should be click-ified?

Work items for ubuntu-14.09:
[stgraber] Ubuntu core livefs image via a new job, for amd64 only: DONE
[cjwatson] define a core-1 framework: INPROGRESS
[jamesodhunt] Fork initramfs-tools-ubuntu-touch as initramfs-tools-ubuntu-core, rename the various scripts and update scripts/touch to support the different partition layout: DONE
[jamesodhunt] Make a new binary package for ubuntu-core configuration and ship /etc/system-image/writable-paths (copy from lxc-android-config): DONE
[jamesodhunt] Rewrite system-image-upgrader in python to work in main FS context: DONE

Work items for ubuntu-14.10:
[mvo] make ubuntu-core-14.10 framework available on core image: DONE
[jamesodhunt] define partition scheme for usb key and VM: TODO
[jamesodhunt] device tarball split out from image: TODO
[sergiusens] add support for this image type into ubuntu-device-flasher: DONE
[jamesodhunt] support system image upgrade (VM): DONE
[jamesodhunt] support system image upgrade (on USB key): TODO
[jamesodhunt] write integration tests for upgrader: INPROGRESS
[jamesodhunt] put cloud-init in the image: DONE
[mvo] click commandline support for talking to the store (lp:~mvo/click/acquire+sso): INPROGRESS
[beuno] resolve requirement for authentication from click commandline on the server to talk to the store: INPROGRESS

Work items:
[mvo] click hook support for systemd unit files (lp:click-systemd): DONE
[mvo] click hook for cli apps (lp:click-bin-path): DONE
[mvo] click support for acquire clicks itself (lp:~mvo/click/acquire): INPROGRESS
[mvo] click support for talk to the server click store (lp:~mvo/click/repository): INPROGRESS
[mvo] click support for sso store access (lp:~mvo/click/sso): INPROGRESS
[cjwatson] work out BIOS+UEFI support: TODO
[jamesodhunt] port ubuntu_command interface to server so that s-i has something to call to apply the update: DONE
[barry] some si work to separate apply-update and reboot steps (D-Bus API changes, etc.) bug #1381538 : TODO
click support to provide "stuff" to other click packages: TODO
[jamesodhunt] prototype mounting the system read-write in a namespace for updates on a running system: DONE
Also include /etc/system-image/archive-master* into that package (? already in system-image-common): DONE
[jamesodhunt] put necessary files in /var/lib/cloud-image to prevent image in IoT mode from looking for external data sources on boot: DONE