Ubuntu Snappy: next steps

CI Gaps: Let's figure out what else we need tested in terms of snappy, Ubuntu Core and related bits.

Deduplication: A snappy package contains all its dependencies including all shared libraries. To avoid storing and downloading identical files multiple times we will use the file hash information we have. On the FS side we can simply hardlink identical files to a place like /apps/.data/ (with multiple subdirs). On the download side we need to get the hashes.yaml from the store first and evaluate this and then only download the files we do not already have. For this the easiest option is to to have a /apps/.data/ directory (probably with a bunch of subdirs so that its not one huge directory).

External dependencies: Let's take a look at snappy's external dependencies and see which of them can be removed.

Core Cleanup: Let's clean up Ubuntu Core, especially the image code and file system.

Agenda:

- Intro

 - Gaps in our test/CI story
   - limited reboot testing right now
   - not upgrade testing
   - no rollback testing
   - no testing of failover features

 - snappifying missing parts
   - kernel
   - OS snap
   - and make them shippable through store
   - use the new type names: OS, kernel, gadget

 - Removing some external dependencies
   - debsig-verify (sign hashes.yaml instead)
   - system-image-snappy-cli (uses store tarball)
   - ubuntu-core-upgrader (move into snappy itself)

 - “deduplication” of content in snaps
   - use hashes.yaml
   - de-duplicated on the FS layer
   - de-duplicate on download (needs server side support)
   - de-duplication of mapped libraries from the linker (difficult, maybe?)

 - Ubuntu Core clean up:
   - make local image building possible -> end goal: one image per commit and run tests on that
   - livecd-rootfs/seeds cleanups
   - /frameworks ?
   - /kernel ?
   - check base image and if we need all parts

- misc:
  - healthcheck

- Snappy shell
  - minimal CLI interface

- Q&A

Notes:
Gaps in our CI story
We got unit and integration tests
Reboot testing is limited
Dont have upgrade testing
No rollback testing
Not testing of failover features
CI:
Smoke tests on a virtualized instance
autopkgtest can not work with ubuntu-device-flash
Goal: image build per commit
Snappy team responsible for the testing itself (architecture, development and so on)
CI integration more at the hook level and infrastructure
We want CI for all snap types, need to sort out the details for that
Need to sort out the details for the technical work that the snappy team will have to do, and then discuss with the CI team how to hook that
Snappifying missing parts
Need a way to hook up image builds with store upload
Also find out to support multiple architectures on the store (multi-part uploads)
Removing external dependencies
right now: click compatibility → long term: use snappy itself
debsign-verify
system-image-cli: becomes part of snappy itself once the OS snap is in store etc. same goes for kernel snap; does not change system-image-clie v3 plans for phone
core upgrader goes same direction
system-image server will move to the store
deduplication
how to deduplicate libraries
store: phased updates need to be supported
canary updates need to be made available for all snap parts in store
healthcheck
needs a complete story/spec :)
principal is to ensure that all snap parts can have health checks; how decision to rollback something that causes cross snap breakage is made is open
Cleanup:
Need to look at the seeds, see if we can make the image smaller
Python 2 and 3 in the image, need to move to python 3 (or not having python at all)
Misc:
Healthcheck when updating applications, to auto rollback to a previous version
CLI interface:
minimal interface to interact with snappy + some additional commands
not bash directly, but something powerful to manage snappy

Work items:
[rsalveti] work with asac, mvo sergio and others to flesh out the current work we have (board), so we could have a base ground for following up stories we need to work on: TODO
[rsalveti] set up the stakeholders team and planning/development process for the next few weeks: TODO