Improve Server Install and Upgrade Testing
The existing Ubuntu Server ISO Tests (see http://
In order to further improve the day-to-day quality of the Ubuntu Server product the depth and breadth of automated and manual testing undertaken during the Q cycle will be improved.
Goals for Q:
Migrate existing tests to UTAH and improve depth and breadth of install and upgrade testing.
Ensure good manual testing coverage at key milestones during development cycle.
Blueprint information
- Status:
- Complete
- Approver:
- Dave Walker
- Priority:
- High
- Drafter:
- Ubuntu Server
- Direction:
- Needs approval
- Assignee:
- James Page
- Definition:
- Approved
- Series goal:
- Accepted for quantal
- Implementation:
-
Implemented
- Milestone target:
-
ubuntu-12.10
- Started by
- James Page
- Completed by
- James Page
Whiteboard
Summary of Discussion at UDS:
Current test cases are OK - but all need to be migrated to UTAH early during the Q cycle.
Some gaps in upgrade testing - members of the server team to identify suitable test cases so that the QA team can setup more regular upgrade testing.
More requirements identified for UTAH - specifically multi-nic and USB install support.
Still need to ensure that we get good manual testing coverage at key milestones - ubuntu-server list will be used to call for testing.
MAAS test cases need some improvement and automation.
Session notes: http://
Assumptions:
- No new tests will be automated using the existing Ubuntu Server ISO Testing Framework.
- All existing tests will be migrated by the Platform QA team as UTAH comes online.
- Existing test format is still valid for UTAH and existing tests can be extended in place.
- The existing upgrade testing framework can support the requirements of the server team.
- OpenStack testing is not covered under the scope of this blueprint.
- Preseed based install testing is still sufficient to prove the ISO install experience.
- Manual testing still has its place a key points in the development cycle.
User Stories: N/A
Test Plan: N/A
Release Note: N/A
Work Items
Work items for quantal-alpha-1:
[smoser] Call for testing of ISO images - Alpha 1: DONE
Work items for quantal-alpha-2:
[james-page] Call for testing of ISO images - Alpha 2: DONE
Work items for quantal-alpha-3:
Call for testing of ISO images - Alpha 3: DONE
Work items for ubuntu-
Call for testing of ISO images - Beta 1 (missed - sorry): POSTPONED
General call for testing of Ubuntu Server Quantal: DONE
Work items for ubuntu-
[ivoks] Expand existing test cases to support different configuration options for mail-server: POSTPONED
[james-page] Work out better more focussed upgrade testing for server tasks: DONE
[james-page] Call for improvement of test cases and coordination with migration to new harness: POSTPONED
[james-page] Migrate existing ubuntu-
[james-page] Review server seeds for gaps in testing of packages in main: DONE
Work items for ubuntu-12.10:
Improve MAAS test cases: POSTPONED
[james-page] Plug the iSCSI tests into UTAH: POSTPONED
Ensure UATH can support USB installation as well: DONE
Ensure multi-nic testing can be supported: DONE
Call for testing of ISO images - 12.10 RC: TODO
[pwlars] Add a profile to auto upgrade testing to support VM upgrade testing (nested VMs): POSTPONED
[serge-hallyn] Provide a test to check the status of the VM after upgrade (nested VM's): POSTPONED
Dependency tree
* Blueprints in grey have been implemented.
