Android Toolchain Benchmarking
Give an overview of what is currently being done for Android toolchain benchmarking in Linaro. Discuss future enhancements we should look into.
Agenda:
* explain what I currently do each month
* discuss some of the short-comings
* discuss some of the future ideas we have and get other ideas on what we should do to improve this
Goals:
* decide what areas we should focus on related to android toolchain benchmarking going forward
* get an idea of the effort required
* does Andy have the time to do this on his own
Blueprint information
- Status:
- Complete
- Approver:
- Andy Doan
- Priority:
- Essential
- Drafter:
- Andy Doan
- Direction:
- Approved
- Assignee:
- Andy Doan
- Definition:
- Approved
- Series goal:
- Accepted for 2011q4
- Implementation:
-
Informational
- Milestone target:
-
connect-q4.11
- Started by
- Zach Pfeffer
- Completed by
- Zach Pfeffer
Whiteboard
More background can be found at:
https:/
Two topics we should discuss are:
1) https:/
2) https:/
[zpfeffer 2011/10/31]
Possible action items:
Fix 0xbench so it runs both manually and automated
Integrate 0xbench and Andy's flow for both manual and automated runtime
Integrate GLmark2 and get FPS
Get SMP benchmarks/test
Integrate CTS
More benchmarks (from Tony)
SunSpider
V8
Vellamo
IE Fishtank (http://
JSGamebench (http://
See https:/
Notes from LDS
Notes
https:/
is generated by a script.
AI: <Android/
AI: Andy: Need to integrate Andy's flow into LAVA
Need to scope
Need to compile on host
ADB
Cross installed (will need to be able to install cross toolchain)
Do we do this as part of the toolchain build?
Test n times each
Frans to help
Need to handle new options
Need a config option
Need to run manually/LAVA
Need to be able to auto gen report current vs flot
What happens when we have a regression?
Month-to-month
Image-to-image
What resources do we have to investigate
Toolchain group can get Skia expert?
perf, 1st level
assembler
gen asm diff of hotspot,
Can toolchain start to use Android's workloads
Look at the history we have
AI: Need to strip down Android image and benchmark
Need way to stop services
Benchmark Bionic?
Just Benchmark Skia
Need to surgicially target specific libs
AI: Alexander to work with toolchain group to help them sort out a basic Android test rig
SSH in
Remote access
AI: LAVA needs to keep an image live
AI: <Android>: Set up remote acess
Need sshd
AI: Andy to sync up with toolchain group about the agreed on benchmark set
AI: SMP?
benchmark.git, cx image change
AI: <Android>: Reach out to benchmark.git
AI: Need LAVA to support parameters in, Yong Qin
