Exploring a hardware database for testing
Discussing the usage of HEXR for testing purposes
Example use cases;
Using your hardware profile inside the qatracker during reporting
Ultimate goal;
Targeted testcases for specific hardware configurations. Testing campaigns to push testcases and results back to developers
Blueprint information
- Status:
- Not started
- Approver:
- Jono Bacon
- Priority:
- Medium
- Drafter:
- Nicholas Skaggs
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Pending Approval
- Series goal:
- Accepted for raring
- Implementation:
- Unknown
- Milestone target:
- None
- Started by
- Completed by
Whiteboard
Demo of HEXR
Will use checkbox to upload submissions
Can see machines and working status of machines, components, etc
Can create testing pools of machines based on any criteria in database
Lacks a 'my hexr' feel -- no way to link specific person to a machine
Lacking true API
Is a tool like this useful? Yes
Do we want this tool? Yes
Not yet open source, but planned to be
Do we want to wait for the open source release or not before adopting? No
Do we want to wait for features we 'need' before releasing? No
Make availible asap, as-is
Is it possible to provide a feature that provide suggestions of drivers and packages might be needed when we do a HEXR upload for a new platform to be enabled? This will be extremely valuable for OEM enablement. This may require integration of database on launchpad and release notes.
yes, driver information is availible
Current HW DB:
Results tracker
ubuntu friendly
hexr
launchpad
Needs/Enhancements:
Can we contact user machines?
API for linking machines, ie the qatracker
Pushing a testing payload to a machine (even turning it on, etc) based on data found in HEXR
Need to create seperate machine instances depending on drivers (so we can be specific when testing as to what is working and what isn't)
Access controlled by single-sign on
Sign in to read? Sign in to edit?
Everybody can read
You can edit your own stuff only
Launchpad team for admin access (can read and write everything)
Agree to make your data availible in HEXR and agree to make your systems available for testing
Old List of Ideas / Needs / Requirements
Needs
Ability to diagnose a user's system and create a hardware profile
How can we enter a custom-built machine (I care more about components than product names)?
seems to use motherboard vendor and name
Can we package up upload-hw.py?
make it depend on dependencies, and simply have users install it and run
Is there a userguide to submitting a system?
can help create if needed
internal website help buttons or links to docs would help
Ability to associate a hardware profile with a specific user's contact information (aka, email address)
I’m thinking linking and integrating with launchpad is probably the way to go here
Would like to NOT have to go to lp to get contact email address; even if machine is linked to user via lp profile
Ability for specific user to opt-in and opt-out (they need to be able to change their mind ) to being contacted for testing
Don’t want to overload with lots of “account” features or place maintenance burden on users
Perhaps make it an attribute that can be changed by the user for their system only
Ability for a specific user to update their hardware profile once submitted
Ability for a specific user to view their hardware profile once submitted
I seemingly have to search for my system by driver or vendor name and drill down and find it. Source isn’t a filterable option; and I can’t select a specific person
Ability for a specific user to link to their hardware profile
for things like bug reports, testing tracker, etc
any API for automagically pulling based on ubuntu sso?
Ability to search the database for users who have opted in for testing and meet my hardware criteria (eg, Atheros AR5007 Wireless LAN adaptor)
playing around with testing pools, I think you can solve this by making a “contactable for testing” attribute, along with including additional information in the export related to the person and links to the system
Nice to haves
Testing pools don’t allow for multiple selection of devices very easily
can I make a search (with all it’s filters, etc) a pool automagically?
Notes
Drivers don’t seem to be useful
data issue only?
can we ensure drivers are a part of the community version?
are they reported now by the tool -- I didn’t see that my nvidia driver version as being reported on my submission
How do we handle updates to a system?
can I see that it didn’t use to work, but now does?
can I tell if something is needed to make it work?
User Stories
The kernel team imports a patch from upstream to enable support for the Atheros AR5007 wireless chipsets. Nicholas searches HEXR for machines containing the chipset and gets a results page showing the current machines containing the device, along with a listing of the machine, person, launchpad profile, and contact email address. Nicholas copies the listing of email addresses from the page and sends an email off directing the users to the testing tracker to test out the new kernel.
A QA community member wants to make his system available for testing. He installs a package and is walked through adding his system to the community HEXR database. He is prompted to allow his system to be a part of the testing pool, and is taken to his submission on HEXR to report compatibility.
A QA community member is contacted about a new package to enable hardware identified in his system. Upon completing the testing, the hardware is now working. The community member updates his hw profile to specify the hardware is now working.
Future Thoughts/Ideas:
How can a user maintain their system inside HEXR?
user account?
package automatically monitoring for hardware changes (what about hardware support changes?)
Is there a need to be able to generically search for hardware that meets certain technologies?
PAE/non-PAE; 64-bit or 32-bit only, sse extensions, vt extensions, etc
Where does ubuntu friendly fit?
Work Items
Work items:
[nskaggs] Talk to mhall119 about creating skunkworks for hexr: DONE
[schwuk] Stand-up new public hexr system for community: TODO
[schwuk] Move over to using checkbox for submissions: TODO
[schwuk] Open source hexr: TODO
[schwuk] Speak with checkbox team to add options to submit to HEXR: TODO
[schwuk] Provide linking and contact information for machine owner: TODO
[schwuk] Provide API call to provide machine information: TODO