Create a mechanism to create a low-variance environment for Android benchmarks
Why?
The Toolchain Working Group has requested that we create a system where the run-to-run variance of our benchmarks is 1% for native and JAVA workloads. These low -variances will allow comparisons across baselines and allow benchmark guided optimization.
Context?
This is part of the larger ARM benchmarking effort.
What gets produced?
A mechanism that runs before a benchmark that puts the system into a low variance state.
Where will the work get put?
A script on android.
Blueprint information
- Status:
- Complete
- Approver:
- Zach Pfeffer
- Priority:
- Medium
- Drafter:
- vishal
- Direction:
- Approved
- Assignee:
- vishal
- Definition:
- Approved
- Series goal:
- Accepted for 2012q2
- Implementation:
-
Implemented
- Milestone target:
-
12.08
- Started by
- vishal
- Completed by
- Zach Pfeffer
Related branches
Related bugs
Sprints
Whiteboard
Notes:
[2012/7/2 pfefferz] Please put notes here.
[2012/7/23 vishalbhoj] We have deployed benchmarking applications into LAVA for snowball and pandaboard. We are observing around variance of less than 1 percent in most of the native tests. Here are the results for pandaboard:
https:/
[2012/7/23 pfefferz] Moving to 12.08. Other Benchmarking tasks took longer than expected.
Meta:
Headline: Native and JAVA benchmarks can be run with 1% variance.
Acceptance: 1. Native variance is less than 1%, 2. Java variance is less than 1%
Roadmap id: CARD-134
Work Items
Work items:
Create script: DONE
Measure native variance: DONE
Measure JAVA variance: DONE
Get feedback from the TCWG: DONE
