Refine our SRU process to be more agile while avoiding too many pitfalls
Our current SRU process is perhaps slightly over-engineered as a reaction to the fact that out previous SRU process was almost certainly far too lightweight. While there's (I hope obvious to all) value in documentation, audit trails, regression testing, and "bake-in" periods, our current process makes it somewhat difficult to get urgent fixes out quickly, while also discouraging the casual observer with an "obviously-correct" 2-character patch from making contributions to stable releases.
We don't want to throw the babies out with the bathwater here, but we need to discuss ways we can make the process more agile while still keeping it robust enough to avoid making serious mistakes along the way.
Blueprint information
- Status:
- Complete
- Approver:
- Steve Langasek
- Priority:
- Medium
- Drafter:
- Adam Conrad
- Direction:
- Approved
- Assignee:
- Adam Conrad
- Definition:
- Approved
- Series goal:
- Accepted for quantal
- Implementation:
-
Implemented
- Milestone target:
-
quantal-alpha-3
- Started by
- Kate Stewart
- Completed by
- Steve Langasek
Whiteboard
We have a very tight SRU process due to a failed SRU. Over time we have adjusted this to adjust to real risk. We are currently saying "no regressions in stable releases", we want to be encouraging "fix things in our stable releases".
PERFECTION for the WIN: all bugs fixed without regressions yesterday!
Problems:
- It is easy to fix things in the development release and we generally don't fix things in stable releases as it is too much paperwork.
- Nobody owns the SRU verification process. It's a volunteer-based process and is ineffective.
- People doing the verification are not generally experts on the subject. People are often having to test their own fixes because "nobody cares".
We question the purpose of the verification process (?).
If there are a lot of affected users, they usually do responsive verification process (or turn the bug into a flamewar)
Institute patch-pilot-like programme to help with SRU verification?
There used to be a verification team but it has dissolved. That team can be resurrected.
Should be quicker to get top crashers on errors.ubuntu.com through verification.
Pre-ping (optionally) to see if SRU is valid.
-proposed installs as a metric (?) not really, doesn't tell you any information apart from maintainer scripts failures as all you know is i have installed it not even run it
* automatic test-cases for SRUs, but about half of them (guesstimate) are not auto-testable
* how does security team deal with sponsoring
-- web of trust: who pushes the patch & where it came from (e.g. debian, upstream, rockstars)
-- main: double verification
-- dedicate team to deal with this time
* can the maintainer test ones own SRU?
- Uploaders should certainly be able to self-validate, if comfortable doing so. If not, use the buddy process.
* how does lp.net team do SRU?
- automatic test suites
- demo test plan on the plan
- manual verification done by the developer fixing the issue
- qa for no regression
- lp.net team lands no-regression commits, even if it doesn't fix the SRU-bug
- qa-untestable tag for exceptionally trivial one-liners (or fixes that cannot be tested)
* whoopsie daisy tracking
- 7 days proxy to be dynamic and be reduced due to possitive verifications
- watch whoopsie daisy trends due to rollouts
- a good list of top10 wanted list
SRU verification queue: http://
Work Items
Work items:
[vorlon] talk to balloons about reviving the SRU verification team: DONE
[vorlon] discuss with stgraber, balloons about QA tracker vs. his tools for gathering this information: DONE
[vorlon] fix SRU procedure to make the "regression potential" section emphasize areas that need additional testing; obsolete "development fix" in favor of bug tasks: DONE
[all] follow up in the phased deployment session about whether we can gather information from whoopsie about -proposed packages that are *not* crashing: DONE
[brian-murray] patch sru-accept to link to .debs for -proposed in comment or the page of the binary package build in launchpad which has all the deb links from the librarian (as the security team does): DONE
[brian-murray] Advertise pending-sru.html in sru-accept comment?: DONE
[dholbach] send pilot scheduling tools to dmitrij.ledkov: DONE
Dependency tree
* Blueprints in grey have been implemented.
