Preparing CEPH for main inclusion
CEPH has become a first class object and block storage mechanism, and has integration hooks for key pieces of Ubuntu Cloud Infrastructure. Especially interesting is CEPH-RBD for storing kvm block devices on the scalable CEPH block store, and the Hadoop HDFS shim to replace HDFS with CEPH in hadoop clusters.
Blueprint information
- Status:
- Complete
- Approver:
- Antonio Rosales
- Priority:
- Low
- Drafter:
- Ubuntu Server
- Direction:
- Approved
- Assignee:
- Clint Byrum
- Definition:
- Approved
- Series goal:
- Accepted for precise
- Implementation:
- Implemented
- Milestone target:
- ubuntu-12.04-beta-1
- Started by
- Robbie Williamson
- Completed by
- Clint Byrum
Whiteboard
Work Items:
[canonical-
[canonical-
confer with kernel-team to decide on which to recommend, ext4/xfs/btrfs (xfs or btrfs, ext4 will not work due to large xattr problems): DONE
define Ubuntu supported configurations of CEPH (only librbd/librados made it into main for precise): DONE
improve ceph charms: POSTPONED
integrate ceph charm with openstack charms: POSTPONED
libvirt changes needed: POSTPONED
MIR ceph ( http://
[serge-hallyn] enable rbd and rados in qemu-kvm builds: DONE
[nijaba] correctly message technology preview for non-supported bits: DONE
investigate apache + 100 continue support for rados S3 GW: POSTPONED
push nagios and collectd plugins into Ubuntu/
setup automated build recipe against upstream source: POSTPONED
import upstream test suite into archive: POSTPONED
[sconklin] ext4/btrfs/xfs testing in QA: POSTPONED
Notes:
qemu-kvm librados builds are blocked on bug 932898 being fix released.
No formal spec needed IMO, this is just a pile of straight forward actions.
* gceph is not reasonable to be in main (upstream doesnt want to support)
Today rbd is loaded loaded at runtime
- MIR librbd and librados for qemu/kvm
- compile qemu-kvm with ceph (rbd, rados) block device support, see:
- https:/
- MIR all of ceph for hosting ceph
- include latest libvirt rbd support (for rbd authentication)
- Existing Openstack Support
- Nova since cactus can use CEPH for block
- Diablo added glance support for CEPH for image storage
- Dreamhost using mainline BTRFS and debian kernel for current deployment
- Dreamhost launching at scale in a few "weeks"
- CEPH is an independent business unit under Dreamhost
- Dreamhost using fastcgi + apache for rados gateway (can use nginx, too)
Dependencies:
modfast-cgi
Concerns;
- Upstream makes a release every 2 weeks
- No stable release branches upstream
- Limited deployment outside Dreamhost
- unencrypted over the wire
- apps can implement security on top of it
- IPv6 is supported
Additional Opportunities:
- Integrate tightly with OpenStack in packaging
- Add support for ceph in juju openstack charms
- BTRFS ideal, XFS also working well, ext4 missing "large xattr" support
- Upstream asserts that CEPH is under heavy development (cmdline programs remind user at every execution)
- in the openstack use case, for kvm, there is no need for kernel code at all
- if we want to support xen, it would require using the block device kernel code for rdb
Support Targets:
- RBD
Work Items
Dependency tree
* Blueprints in grey have been implemented.