Discuss tests for kernel SRUs

There are already some kernel-specific tests available:

- The autotest suite from http://test.kernel.org, with around 70 different tests (the actual total is 89, but some are examples, and internal consistency tests). Autotest is not packaged, and packaging it might be a complex task: many of the tests require external sources at specific versions (in the Autotest SCCS, usually as a compressed tar); additionally, some of these external sources have local (to Autotest) patches. Evaluation of autotest and ltp has shown that autotest is a better option for our automated testing, and further work will be done on autotest. Packaging is not required in order to continue using autotest.

- the Ubuntu QA Regresssion tests, with about 5 different tests directly related to the Linux kernel. These are also not packaged.

[gema] Are we already running http://ltp.sourceforge.net/documentation/test_plans/linux_test_plan.php ? They seem quite interesting in terms of exercising the kernel. Even the database test cases would add a good amount of coverage to the kernel in particular and to the file system. Apologies if they are the same as test.kernel.org, but the page is down for maintenance at the moment.
[sconklin] The tests referenced above use ltp, which we are no longer pursuing. However, there are a number of good stress tests and tests which exercise the kernel in autotest, and I have begun looking at those. I believe that they provide similar value to the ltp tests and are more maintainable.
If we find tests from ltp that we feel we must have, we can port them to run under autotest without much work.
[gema] Another thing we could consider is turning the stress testing included in ltp into functional test cases, since they seem to be quite fundamental and would run in less time if we turn them into functional tests. I still have to look into them in detail, my comments are just based on their specification.
[sconklin]
Since the QA frame work won't be complete enough to land production kernel SRU tests in during this cycle, I will be building a test sandbox using kernel team resources, so I can sandbox tests and
continue to generate test cases. These should drop into the jenkins framework in QA when it is ready.

Work items for precise-alpha-1:
[sconklin] get rteval packaged, along with anything it depends on: POSTPONED
[sconklin] Provide application to log data from Fluke meters over network: DONE
[sconklin] Compile a list of tests other than ltp which are available: DONE
[sconklin] Evaluate which tests outside of ltp we should be running: DONE
[sconklin] Evaluate rteval and stress and other tests which may provide metrics for trending across releases: DONE

Work Items for precise-alpha-1:
[sconklin] get latest version of autotest into use: DONE

Work Items for ubuntu-12.04-beta-1:
[sconklin] Create a local sandbox using kernel team resources for test development: DONE
[sconklin] Create a way to track test metrics across repeated tests: DONE
[sconklin] Evaluate which additional tests from autotest we should be using: DONE
[sconklin] Create a test for ext4 file systems tests: DONE
[sconklin] Create a test for the Bonnie+ file system stress test on ext4: DONE
[brad-figg] Create a test for the ecryptfs file system tests: DONE
[sconklin] Create a test for the Bonnie+ file system stress test on ecryptfs: DONE