Server Load Testing Suite
As part of server testing, and specially introducing the ARM server release, a set of standard load/stress testing needs to be introduced. Covering the basic areas (memory, cpu, I/O ,..) but also the most common server applications:
Distributed Datastores
“LAMP” Web Server
Caching
Backup Server
This test capability should also be considered for Certification testing and benchmarking
It is also important to understand that this could require significant outlay in capital for new hardware. Depending on what we wish to focus on, we could need to purchase a server or two to serve as full time targets, storage solutions such as Fibrechannel or iSCSI, 10Gb Ethernet devices, fibre cables, network cables, PDUs, KVMs and so forth.
Additionally, it is my thought that Hardware Certification should focus on tests that test the hardware subsystems themselves, and Platform cover the more QA related tests, such as testing various common applications like the LAMP stack, Squid, mail services, and so forth.
The Definition of Done as listed below in the whiteboard is lengthy for now, but lists very specific items that I believe need to be achieved in order to do worthwhile server testing. In the end, the true goal is to be able to really hammer servers in various situations to ensure that 12.04 becomes the best LTS release for servers ever.
The full spec url includes user stories for both the greater server test suite expansion and the load testing subset described here.
LP Wont let me use the same url so pasting it here:
https:/
Blueprint information
- Status:
- Not started
- Approver:
- Robbie Williamson
- Priority:
- High
- Drafter:
- Jeff Lane
- Direction:
- Approved
- Assignee:
- None
- Definition:
- Approved
- Series goal:
- Accepted for oneiric
- Implementation:
-
Deferred
- Milestone target:
-
ubuntu-11.10
- Started by
- Completed by
Whiteboard
Work Items:
[canonical-hw-cert] Jeff to list the testing team info so interested parties can sign up and participate in making this happen: DONE
[canonical-hw-cert] Create a list of the tests to be run (a small number of useful tests to start, we can expand afterwards).: POSTPONED
[canonical-hw-cert] Documentation of test cases and configs and other useful information (wiki?, testcases.
[canonical-hw-cert] Package a consistently runnable test suite: POSTPONED
Previous discussions:
https:/
https:/
Notes from this session:
http://
Q: Can you please try to address IPMI (Server management) as well
A: To what extent? There is already an IPMI test that performs basic function testing (e.g. connect to the BMC, pull SEL events and system info to ensure that the IPMI bits are working).
Definition of Done:
1: Defined list of specific hardware subsystems on servers that are to be tested
2: Defined list of specific common server applications that are to be tested
3: Scope infrastructure changes as may be needed (new hardware, etc)
4: Defined list of tests that will verify server stability and functionality
5: Defined limits on what is acceptable load minimums and maximums for testing
6: Defined list of which tests will be hardware certification tests and which will be QA style tests
7: Responsibilities for each set of tests assigned to the appropriate teams
8: Tests written if they do not currently exist, tested for usability and implemented in Checkbox
9: Separate whitelists defined in checkbox for server certification and server QA
10: New infrastructure in place and operational if necessary
11: Servers tested using full server certification and server QA whitelists
12: 11.10 is thoroughly hammered using our new combined Super Monkey Powers
13: 12.04 is the best Server LTS EVER!
Work Items
Dependency tree
* Blueprints in grey have been implemented.
