Containers and host/client model for Android + Ubuntu

Currently Android is the first and main host OS for the Touch project, with Ubuntu living inside a container (the communication between both happens via hybris, binder or socket).

The goal of this session is to discuss if this model should be continued, and/or identify possible alternatives. The most desired solution, from the maintainability point of view, is to have Ubuntu as main host and with the Android isolated based on what is needed and used by the system (Android is needed so we can reuse the Hardware Abstraction Layers).

Topics
 Present the current architecture and why it was used
 Discuss the viability of having Ubuntu as the first and host OS
 Identify which components of Android need to be maintained and how to better bridge that into Ubuntu (probably via a container, but a shared/mixed system might be doable as well).
How to handle udev vs. ueventd, vold and netd

Notes:
Android is the first system (OS) booted, bringing service flinger, audio flinger, media server and a few other specific android services.
Later in the boot ubuntu_chroot is called (uchroot), which starts a container where Ubuntu is hosted (/data/ubuntu).
upstart launched in container to start Ubuntu layer in Ubuntu rootfs

The android part is needed so ubuntu could reuse the hardware abstraction layer (and a few core services as well).

System updates
Want to update Android and Ubuntu bits at the same time to avoid mismatches
Could leverage APT if Ubuntu was toplevel

Security reasons
AppArmor profiles would be needed for both sides instead of one, with different layouts
    do you plan on incorporating AppArmor into android or did I misunderstand this? we are using AppArmor on Ubuntu side and would like everything running to be covered by AppArmor profiles

Two attack vectors if we keep two hierarchies

Using libhybris: can only use from Ubuntu side right now; would want to use from Android side as well.

Currently nuking udev from Ubuntu chroot which causes a range of problems (as we're sharing the same /dev).

== Issues with Android root and Ubuntu Container ==
 * Can't start udev
 * Security confusion
 * Can't upgrade kernel with dpkg (not necessarily true, as the kernel is part of the boot img - initrd)
 * Two init's might setup conflicts
 * Difficult to synchronize libhybris and platform-api versions (as we need libraries available at both sides)

== Advantages of Android root ==
 * We know it can work
 * A working boot img is create during build process (ubuntu would need something else to handle the booting/mounting the fs before calling upstart)

== Why we need a container ==
 * Two libc's might be confusing with LD paths, etc.
 * Proprietary binaries and services might use the same path which is used by ubuntu, such as /etc and /usr (could be solved with Apparmor path rewriting) (Probably easier to simply put them in a container)

It is too early to try to merge the two roots given the number of Android bits we're reusing

Could we run upstart as PID 1 in the Android side only?
Problem is that Android init provides properties to other Android bits

Is it hard to flip Ubuntu and Android roles? (Ubuntu first)
Might be hard to patch Android to run in containers
Currently not shipping system image, only patching it a bit (android is available at /system)
Rely on having separate partitions to be able to switch Ubuntu and Android hierarchies via boot.img
Can't fit Ubuntu in system partition with Android today

Can't poke at partition layouts before we target our own devices

Why are we sharing /dev? Because Android setups weird devices that we need

Could we leverage standard LXC helpers instead of our custom container wrapper? Likely yes, but would need some tweaks and would need to include the LXC helpers in Android side

Work items mostly handled by awe, mfrey and rsalveti

== Device Specific Configuration ==

Some discussion ( awe, ogra, rsalveti ) happened recently after Bluez was added to our Ubuntu images. Initially brcm-patchram-plus-nexus7 was added to the images, however this only applied to Nexus 7s. This change has been backed out, and for now this will be handled on the Android side.

We discussed whether we'd need a mechanism similar to oem-config that could potentially look at device specific attributes ( maybe Android build property values ) and disable system and/or session upstart jobs ( via override files ), remove packages, ...

This also applies to session jobs as well ( see /etc/ubuntu-session.d to session device specific configuration files for GRID_UNIT_PX, DPR, ... ).

Work items for ubuntu-13.05:
[mfrey] Investigate the effort about moving the android system in its own container (and having ubuntu as the first os): DONE
[rsalveti] Make android ramdisk for consumption with LXC available from android image in /data/ubuntu/boot: DONE
[ogra] If Ubuntu is the first host, work is needed to make a functional boot img with ubuntu specific initramfs: DONE
[ogra] Make sure to keep partition layouts / storage facilities (cache/userdata/system) after container flip: DONE
[ogra] Package adbd and make it automatically start on boot after the container flip: DONE
[ogra] Implement automatically starting LXC container for android: DONE

Work items for ubuntu-13.06:
[ogra] create proper device specific zip file via cdimage (requires android toolchain and "update-binary" in the archive): DONE
Investigate ueventd vs. udev with the current container model: DONE
[ogra] Research system wide upgrades options: DONE

Work items:
Investigate oem-config like mechanism for device-specific configuration: TODO