Android LEB Validation and Test Suite

Registered by Zach Pfeffer

This discussion will focus on LEB validation and test suites. Topics will include: Monkey, CTS, Monkeyrunner, unit-tests, continuous integration merge tests.

Discussion will include:
1. Now we have basic, monkey, 0xbench, cts, on the going, What more validation should we have for Android?
2. The requirement for validation framework on Android.

Possible Action Items from the discussion:

1. Integrate Monkey.

2. Integrate Monkey Runner and tests that use it.

3. Integrate tests from partners.

4. Create and integrate low-level tests.

5. Create and integrate native-level tests.

6. Create a list if things we need to validate and test.

7. Create a way to summarize and communicate test results across platforms.
7.1 Create a visualization mock-up for Paul.

8. Export tests for local execution.

9. Write tests using LTT and perf.
9.1 Integrate LTT on Android.

10. Run CTS
10.1 Break CTS up for debug and test parallelization.

11. Define and setup peripherals we will test with.


Session notes:

Blueprint information

Not started
Zach Pfeffer
Jeremy Chang
Needs approval
Jeremy Chang
Series goal:
Accepted for 2011q2
Milestone target:

Related branches



Overall Goal:
The goal of this Blueprint is to enhance validation for all builds. This includes adding more tests, making tests run more efficiently, communicating test results and making test modification easy. Linaro has an opportunity to shine with our validation offering as most community projects fail to run or document the quality of their builds.

Work items (month 1):
[pfefferz] Create a list of things we need to validat and test: TODO
[jeremychang] Integrate monkey into 0xbench: TODO
[jeremychang] Write a test that starts a browser, goes to and quits using Monkeyrunner: TODO
[jeremychang] Integrate browser test into 0xbench: TODO
Write other monkeyrunner tests (many tasks - will expand): TODO

Work items (month 2):
Find and integrate tests from partners (many tasks - will expand): TODO
Create and integrate low-level tests (many tasks - will expand): TODO
Create and integrate native-level tests (many tasks - will expand): TODO
Create tool-chain tests: TODO
Integrate and run CTS: TODO
Break up CTS and identify parts that can run individually: TODO

Work items (month 3):
Create a test summary visualization mock-up for Paul: TODO
Document and enable all tests locally: TODO
Define peripherals we need to test with: TODO
Setup peripherals: TODO

Work items (month 4):
Integrate LTT into Android kernel: TODO
Adapt LTT client for Android: TODO
Integrate LTT into validation: TODO


Work Items

Dependency tree

* Blueprints in grey have been implemented.