15.04 migration from Upstart as system init (PID 1) to systemd

= Summary =

Although this topic has been discussed before (see References below), this cycle we fully intend to make the switch to systemd as PID 1.

= Remaining services that need porting to systemd =

See: http://people.canonical.com/~jhunt/systemd/packages-to-convert/ (updated daily) and now summarises those packages that only have upstart configurations (but no sysv and no systemd configuration).

= systemd Quick Start =

It is already possible to boot a desktop vivid system using systemd since, currently, both upstart and systemd are installed. Either modify the kernel boot line in grub to specify 'init=/bin/systemd', or "sudo apt install systemd-sysv".

= Wiki =

Please consider updating the page below with any new information you find:

https://wiki.ubuntu.com/SystemdForUpstartUsers

= References =

 * https://blueprints.launchpad.net/ubuntu/+spec/core-1403-systemd-transition

= See Also =

 * https://blueprints.launchpad.net/ubuntu/+spec/core-1403-session-init-systemd-migration

== Session notes ==

Where we are:
Still discovering a few bugs. Debian switched a while ago. Pitti has had it running on his desktop for a few months and it seems fine.
Work required on server & phone, mainly testing at this point.
Generally we are ready to switch now.
How do we organise the change over? A (virtual) sprint would be useful.
There is a wiki page (https://wiki.ubuntu.com/SystemdForUpstartUsers) which can help you to convert jobs from upstart to systemd.
not committing to supporting 3rd party user jobs; if we do, we need upstart "deputy init"
we can possibly replace all the events that the session jobs expect with other means like inotify

Blockers for switching the default in vivid:
* The packages which dont have systemd jobs (http://people.canonical.com/~jhunt/systemd/packages-to-convert/)
* AppArmor - needs fixing before systemd becomes the default (#1350947)
* lxc for user containers (needs different cgroup handling, #1346734)
  - will patch systemd for this in Ubuntu: have systemd create all of the cgroups, make cgmanager work on systemd
* fix LXC/systemd to run in a container guest, SRU LXC to trusty to support vivid containers with systemd as pid 1 (#1347020)
* minimally working cryptsetup integration to be able to boot
* juju creates upstart jobs and needs to support creating systemd units

Blockers for release:
 - network bringup (ifup -a not sufficient); there's also ifup@<interface> for hotplug events, but that mostly works for physical ifaces
  - virtual interfaces (bond, vlan, bridges) in /etc/network/interfaces do not work. E. g. /e/n/i might define a bridge for "eth1.5000" which only appears later on during runtime, thus isn't covered by "ifup -a". Thus ifup@.service needs to bring up not just the hotplugged interface but all of its reverse dependencies.
  - implementation of the network failsafe?
  - auditing/testing of ifupdown hooks for systemd compatibility
 - NFS /home - and nfsroot (casper)
 - make sure friendly-recovery and emergency mode work
 - fix fsck plymouth integration (#1316796)
 - check cryptsetup/plymouth integration, fix
 - check behaviour of a file system which isn't available from fstab; -> recovery shell, mandatory/optional mounts; also handle mountall specific userspace mount options (wait/no_wait, etc.)
 - check mobile kernels to make sure systemd works everywhere
 - CPC cloud images have some dynamically-generated upstart jobs that will need reimplemented

Nice-to-haves:
 - add apport hook for collecting useful information (look at Debian reportbug hook + upstart hook)
 - precise: better to do a clean backport of current LXC
- Add autopkgtest for booting with upstart (init=/sbin/upstart)
- extend DEP8 tests (minimal ones are there)
- add grub entry to boot with upstart
- automatic fallback to upstart in grub (additional boot flag/checkpoint)

Documentation Requirements
* How do you convert an upstart job to systemd unit
* How to debug issues: boot failure, shutdown hang, failure of a particular unit
bugs specific to booting with systemd: https://bugs.launchpad.net/ubuntu/+bugs?field.tag=systemd-boot

== ifupdown hooks review ==
checked all packages with ifupdown hooks: zgrep network/if- Contents-amd64.gz | awk '{print $2}' | sed 's!^.*/!!' | sort -u|xargs

most packages are init script agnostic and either don't involve services at all, properly use invoke-rc.d, or are systemd aware (like openvpn). Some (postfix) call the init.d script directly, but our LSB snippet diverts it to the right init system. Only mythtv-backends needs fixing (bug 1408651).

The upstart hook properly handles running under systemd, but we need to audit consumers of the net-device-up and static-network-up events. This is really a part of porting upstart jobs to systemd units or init.d scripts tough.

== Doing the switch ===

Changing ubuntu-meta seeds from "upstart" to "systemd-sysv" should be sufficient (with recently fixed ureadahead). This has been done in https://launchpad.net/~pitti/+archive/ubuntu/systemd, and tested with an upgrade from trusty and utopic.

Work items for ubuntu-14.12:
[pitti] test cryptsetup+LVM boot (ubiquity default layout) with systemd (works fine as expected: this happens in initramfs where there's no init system difference): DONE
[pitti] test encrypted non-root partition (e. g. /home only) with systemd (no cryptsetup in initramfs; works fine): DONE
[pitti] add apport hook for collecting useful information (look at Debian reportbug hook + upstart hook): DONE

Work items for ubuntu-15.01:
[vorlon] Plan porting sprint: DONE
[pitti] check behaviour of a file system which isn't available from fstab (behaves as specified: waits for 90s for it to appear, then drops to rescue shell): DONE
[pitti] handle mountall specific userspace mount options (bootwait/nobootwait, optional, showthrough): DONE
[pitti] audit/test ifupdown hooks for systemd compatibility: DONE
[didrocks] add grub entry to boot with upstart: DONE
[didrocks] Add support for fallback X session: DONE
[pitti] Add autopkgtest for booting with upstart (init=/sbin/upstart): DONE
[pitti] add static-network-up compat (integrate into network-online.target): DONE

Work items for ubuntu-15.02:
[xnox] port away from events-bridge / :sys:* events usage: DONE
[xnox] port upstart-local-bridge to be compatible with systemd: DONE
[pitti] prepare/test upgrade with seed change/ubuntu-meta from trusty and utopic: DONE
[pitti] port ureadahead to systemd: DONE

Work items for ubuntu-15.03:
[pitti] test/fix bringup of virtual (bond, vlan, bridge) net ifaces that depend on hotplugged ifaces: TODO
[pitti] test NFS /home (needs #1312976 fixed first): DONE
port dynamically-generated upstart jobs in CPC cloud images: TODO
[pitti] switch to systemd by default in "init" and ubuntu-standard (FFE bug 1427654): DONE
[pitti] make seed and other changes to keep upstart on touch (#1428026): DONE

Work items:
check mobile kernels to make sure systemd works everywhere (too old, won't switch to systemd on touch): POSTPONED
[xnox] support globs in the upstart-local-bridge under systemd: TODO
port ureadahead bits from lxc-android-config for other mount points: TODO
[apw] merge initramfs-tools (not a blocker for systemd, and 0.108 caused some regressions in Debian): INPROGRESS
[vorlon] merge console-setup (not a blocker for systemd): DONE
automatic fallback to upstart in grub (additional boot flag/checkpoint): TODO
test once console-setup & initramfs-tools got merged that separate /usr mountpoint boot works: BLOCKED