Reporting the state of quality in ubuntu
NOTE: This has no relation to the current Weather Report :-)
Create a realtime report that will convene the 'quality' of the ubuntu archive at any point during the development cycle. Summaries from manual and automated testing should be included, as well as pending releases and manually targetted ares to focus.
Blueprint information
- Status:
- Not started
- Approver:
- Jono Bacon
- Priority:
- Essential
- Drafter:
- None
- Direction:
- Approved
- Assignee:
- Nicholas Skaggs
- Definition:
- Approved
- Series goal:
- Accepted for raring
- Implementation:
-
Unknown
- Milestone target:
-
ubuntu-13.04
- Started by
- Completed by
Whiteboard
[mpt] Why are the QA Dashboard, the QA Weather Report, and the Defects Dashboard three different things? That would make it less likely that people would know about the ones that would help them.
During the course of the cycle it is difficult to understand the quality of the development version of ubuntu. While it can sometimes be easy to know of critical bugs (because you and everyone else running ubuntu+1 are experiencing the error :-) ), in general problems often creep up unnoticed, or are not realized until too late. In addition, it is sometimes difficult to know when and how to foucs testing efforts, both on a coordinated and individual basis.
Potential Data Sources:
QA Dashboard
Reports like:
http://
or launchpad directly
Jenkins
Weekly Release Meeting Summary
Community Feedback (mailing list, forum, IRC, bug reports)
QATracker
errors.ubuntu.com / whoopsie dasiey?
Package status reports:
http://
Lintian (http://
Piuparts (not sure if we're running this ATM)
Debcheck (http://
Report Features
Packages needing focused?
confidence level and thumbs up / thumbs down
see: work: https:/
test and coverage of critical packages.
how well exercised the packages are - bugs only make sense in context of how exercised the package may be.
Per package level reporting
Per image reporting
correlate package level data to images?
get an initial scale..
include archs
Correllated image - plars, kate.stewart interested in core set of images.
Paul is interested in thresholds rather than absolute.
Manual side to occur on weekly basis, daily for automated.
Use QA Dashboard, not wiki
Work Items
Work items for ubuntu-
[nskaggs] Launch initial version of contributions statistics for manual and automated testing: DONE
[nskaggs] Integrate initial work onto QA dashboard: INPROGRESS
Work items:
[nskaggs] aggregate daily status from jenkins automated tests per image per architecture as a page - red/yellow/green? other? into a prototype.: POSTPONED
[nskaggs] broadcast prototype around and call feedback session early in cycle.: DONE
[nskaggs] review work from Q cycle, and determine what might be applicable.: DONE
