QA tasks for 2013
A long list of automated QA that we should build during first half of 2013 / before 3.0 GA. If you want to use launchpad blueprints to write more detailed plans, please create new blueprints and make them a dependency / sub-task to this one.
Blueprint information
- Status:
- Not started
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Henrik Ingo
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- New
- Series goal:
- None
- Implementation:
-
Unknown
- Milestone target:
-
3.0-beta
- Started by
- Completed by
Related branches
Related bugs
Sprints
Whiteboard
Work Items
Work items:
*********Automated testing*
Setup Jenkins in AWS: TODO
Jenkins slaves should be run on-demand to save money / enable more heterogenous slave pool: TODO
Jenkins is GUI based, but can we store the configuration in svn/bzr: TODO
*********Automated packaging*
Automated rpms, debs, tars from Jenkins: TODO
*********Platform support matrix*
Make sure sufficient tests are run on all supported operating systems: TODO
Actually testing all the different installation packages (tar, rpm, deb...) and not just "make test" from source tree: TODO
Test things that users actually do, such as "service mysql restart", cluster bootstrap with various methods: TODO
Test upgrade path for each of these, rpm->rpm, tar->tar, etc: TODO
Rolling upgrade can be postponed until we are better at automating clustered tests?: POSTPONED
*********MySQL Test Run for single node***
Disable all tests that don't work, and start routinely run what is left as regression test suite: INPROGRESS
Make all MTR tests work and enable them back, when wsrep_on=off: TODO
Make all MTR tests work for single node with wsrep_on=on: TODO
*********MySQL Test Run for multiple nodes**
To be decided how we run tests for clusters, candidates below: TODO
MTR can run multiple mysqld on localhost, could use same approach for wsrep: TODO
kewpie: TODO
ANSTE cluster testing framework, see http://
something else: TODO
first run same MTR tests against single master, other nodes slaves: TODO
also find out a way to make queries with multiple nodes: TODO
*********Test coverage*
Go through all old bugs and make sure there is a regression test: TODO
Create test plan (ie a written document) that lists all major use cases that we claim to support (mysql replication compatibility, 5.6 gtid, foreign keys, etc...): TODO
Implement functional tests for those use cases: TODO
*********More tools**
Run tests (mtr) under valgrind: TODO
Run tests with lcov: TODO
Check with MariaDB and Percona if there's something they use that is not yet listed here: TODO
*********Cluster robustness tests**
Using same framework as picked above? Test killing a node, network disconnects, split brain, etc: TODO
*********Random Query Generator*
*******
For example, setup a Drupal stack against Galera, in multi-master configuration, and run the Drupal test suite: TODO
Find other open source apps that have good test suites and do the same, for example e-commerce: TODO
Maybe even some closed source apps could give us a license for the same: TODO
*********Benchmarks as regression tests**
sysbench: TODO
DBT2 or percona-tpcc: TODO
Could we have something nastier?: TODO
Drizzle test suite includes a tool that will compare a sysbench and DBT2 run against previous results and return failure if results are more than 5% lower than previously: TODO
