Next steps for Hadoop on Ubuntu
Rationale:
During Ubuntu precise, packages for hadoop, hbase, hive, zookeeper and pig where delivered via PPA for use in associated charms which are now avaliable in the charm store for Ubuntu 12.04.
This work needs to be continued to provide support for new versions of hadoop (and family) and to add new features to the charms avaliable in the charm store.
Goal:
Add new features to existing hadoop charms including kerberos support, hadoop-2.0 series and support for Apache Bigtop on 12.04.
Blueprint information
- Status:
- Not started
- Approver:
- Antonio Rosales
- Priority:
- Medium
- Drafter:
- Ubuntu Server
- Direction:
- Approved
- Assignee:
- James Page
- Definition:
- Approved
- Series goal:
- Proposed for quantal
- Implementation:
-
Deferred
- Milestone target:
-
ubuntu-12.10
- Started by
- Completed by
Whiteboard
UDS Discussion Topics:
Quick 101 on current packaging and general approach.
- Based on binary distributions from upstream
- Re-use of Bigtop packaging where appropriate
- Rebuild of native components (to support 12.04 and ARM).
Alignment to next release of Apache Bigtop and potential collaboration (Ubuntu 12.04 will be the targetted release for Bigtop 0.4.0).
- Stick with Hadoop 1.x or move to 2.x (former 0.23) branch (or both).
- Process and technology for feeding back patches to Bigtop (not easy ATM).
- Work to support Ubuntu 12.04 as target version for Bigtop.
Updates to stable packages for Ubuntu 12.04
- Release/Testing process
OpenJDK7 as default Java in 12.10 - support from upstreams and testing process
Revisit packaging from source.
Testing of packages and functionality
Enable kerberos authentication in the current charmset?
Use of puppet modules in juju charms.
http://
User Stories:
Toby want todo some testing of hadoop-2.0 prior to upgrading his existing hadoop-1.0.x deployments. He is easily able to deploy a hadoop-2.0 cluster using Juju and the hadoop charm from the Juju Charm Store.
Frank likes the idea of using Juju to deploy hadoop but he's unsure about the quality of the charms. He can easily see the depth of testing being undertaken on each of the charms and decides that this is the right choice for his deployment.
Janice wants to deploy a secure hadoop cluster. She is able to leverage the hadoop charms inconjunction with kerberos to support this requirement.
Assumptions:
- hadoop-2.0 will be packaged alongside hadoop-1.0.x (different PPA or different pkg names).
- Hadoop will run OK on openjdk-7
- Suitable resources for regular testing of hadoop charms.
- Changes can be fed back to bigtop easily
- All work will also target Ubuntu 12.04 as well as 12.10
- No new releases of components/charms for Ubuntu 11.10.
- Packages will continue to support ARM based architectures.
Test Plans:
Charms will be automatically tested.
Release Note:
N/A - still outside of the main archive.
Work Items
Work items:
Update bigtop-utils and hadoop charms to support openjdk-7: DONE
Testing of existing hadoop charms with openjdk-7: DONE
Update hadoop charms to support kerberos based authentication: POSTPONED
Review use of puppet modules in hadoop charms: DONE
Write/update charms to use bigtop 0.4.0 release repositories: INPROGRESS
Setup automated testing of charms using bigtop tests: POSTPONED
Support usage of Oracle EJRE for ARM based deployements for hadoop 1.0.x: INPROGRESS
Dependency tree
* Blueprints in grey have been implemented.
