Generate debug symbols for all package versions

Registered by Evan

We currently only generate ddebs for the most recent versions of packages in the archive. With the new crash database, we receive a lot of core dumps which we cannot completely retrace because of this issue (https://errors.ubuntu.com/api/retracer/results). For the Launchpad-reporting side of things, this results in an ugly "cannot report this problem" dialog.

IS is also not happy with the manner in which this is currently implemented.

Let's get ddebs support into soyuz.

Blueprint information

Status:
Started
Approver:
Steve Langasek
Priority:
Medium
Drafter:
Evan
Direction:
Approved
Assignee:
Evan
Definition:
Approved
Series goal:
Accepted for raring
Implementation:
Started
Milestone target:
milestone icon ubuntu-13.04
Started by
Kate Stewart

Related branches

Sprints

Whiteboard

Considerations:
- What's the state of ddebs in PPAs?
- https://errors.ubuntu.com/api/retracer/results

IRC discussion from January:

[15:18:08] <ev> pitti: for the "Stop cleaning up ddebs after 7 days" task in support of the crash database, is it just a lack of disk space preventing us tweaking this, and thus a conversation I should have with James?
[15:18:52] <pitti> ev: yes, pretty much; we are running into ENOSPC all the time, and already had to kill many arches/releases from ddebs.u.c.
[15:19:05] <ev> okay
[15:19:10] <ev> I'll go beg then
[15:19:22] <pitti> ev: we recently got some more space on that machine, I already sent an emergency RT a few weeks ago
[15:19:53] <pitti> ev: I can certainly bump it a bit, though; how many days do you need?
[15:20:44] <ev> well, I'd like to see what the upper limit on their side is, as, and correct me if I'm wrong, this is what's causing us to have to mark slightly out of date crashes as invalid
[15:21:24] <pitti> ev: well, not only that; apport-retrace currently uses python-apt to retrieve the debs, i. e. it needs package indexes
[15:21:35] <pitti> and our Packages.gz only has the most recent version
[15:22:31] <pitti> ev: so it'll require some more work in ddeb-retriever to figure out in which Packages.gz an old version should go into, and then some version selection in apport
[15:22:44] <pitti> so far there was not much pressure to do that, since we couldn't even fit all our current ddebs
[15:23:22] <ev> pitti: okay, thanks for the clarification
[15:24:10] <pitti> ev: and we don't only need the ddebs, we also need the regular .debs in their old version
[15:24:40] <infinity> pitti: I take it we're still stalled on ddebs in soyuz? Cause it has all the logic required to do these things, I suspect (inculding keeping old versions in the index for a while).
[15:24:45] <pitti> ev: at that point I guess it's easier to stop using Packages.gz and just directly look up the .deb/.ddeb on the mirror, assuming that they use a standard directory layout
[15:25:17] <pitti> infinity: I haven't heard any update about that in a while, so I take it it's "yes, still stalled"
[15:25:21] <ev> ah, okay
[15:25:58] <infinity> pitti: Maybe we should revisit it sometime. See what still needs to be done to make it work, identify some bite-sized chunks, and either work on it or escalate.
[15:26:15] <infinity> pitti: The "temporary" ddeb infrastructure is a bit past its prime. :P
[15:26:35] <pitti> infinity: nice understatement :)
[15:27:02] <pitti> nothing ever lasts as long as a stopgap solution

(?)

Work Items

Work items:
[ev] teach apport to fetch ddebs from the librarian as a fallback from python-apt: TODO
[ev] teach apport (install_packages) to fetch binaries from the PPA origin (in the Packages field) for the package under apport-retrace: TODO
[ev] rework "cannot report" dialog text with mpt to clarify that we're still sending to daisy (implement after fixing #957177): TODO
[adconrad] Discuss switching to a dual-publisher approach for archive/ddeb with wgrant and webops (for disk space and performance issues on ftpmaster; may create fdt issues?): TODO

Dependency tree

* Blueprints in grey have been implemented.