Ubuntu

Speed up downloads of .debs by using apt-sync

Registered by to be removed

http://lwn.net/Articles/329484/ describes deltarpm, a system to use the rsync algorithm to speed up downloads of RPM packages. A similar system might be quite helpful for .deb based systems, and zsync seems like the perfect tool for this. Proposal: a) design a benchmark to measure usefulness of delta downloads b) extend zsync to understand .debs c) extend apt to use dpkg-repack and .zsync files to download only deltas and generate an identical copy of the new .deb d) run benchmarks. (Not included: modify archive maintenance software to generate .zsync files. That can be done later, if the benchmarks prove this is useful.)

Added 2009-05-29: apt-sync, by Felix Feyertag, should already implement this. This needs to be a) tested b) benchmarked c) integrated to the archive, if at all possible.

Blueprint information

Status:
Complete
Approver:
Colin Watson
Priority:
High
Drafter:
None
Direction:
Needs approval
Assignee:
Canonical Foundations Team
Definition:
Superseded
Series goal:
None
Implementation:
Deferred
Milestone target:
None
Completed by
Robbie Williamson

Related branches

Whiteboard

This would really benefit people connecting over 3G, who want to do a major package update or dist-upgrade, but don't want to pay a "bazillion" dollars for the data downloaded. Also people in 3rd world countries where bandwidth costs up to $10 per gigabyte or even more.

No spec, so adding here. I've seen three possible use-cases for diff updates:
a. User running Ubuntu in developing nations where they have poor connectivity. So the size of updates impacts them. A diff update would reduce the amount they have to download and therefore how easy it is for them to stay updated.
b. User on a bandwidth connection where they pay for bandwidth (3G). They want the smallest possible size of update to reduce cost.
c. Situation where you want to be able to revert an update. An example of this is an appliance where you apply the update, it doesn't work and then you want to revert back to the previous position. Not sure if this specification provides this capability.

2009-06-15 cjwatson: approved with some inline comments

2009-06-19 cjwatson: liw, could you nominate a target milestone for this that you think would be feasible?

2009-10-01 robbie.w: This past week liw had been playing with debdelta-upgrade, but various issues meant he had not been able to get it to work yet. We're now in karmic-bug-fixing mode, so have to defer for potentially lucid, unfortunately. The good news is that debdelta does work, and its author seems to want to integrate it with apt.

2009-10-21 f.feyertag: I've created some benchmarks for apt-sync upgrade and dist-upgrade:
https://wiki.ubuntu.com/AptSyncInKarmicSpec/JauntyBenchmark
https://wiki.ubuntu.com/AptSyncInKarmicSpec/KarmicBenchmark

2009-11-12 robbie.w: I don't think we can get this into Lucid. Too risky for an LTS and we'll not have the resources available in the Foundations team to do the work. With that said, it could be worth discussing what could be done during Lucid to be better prepared to implement in Lucid+1

2011-04-22 robbie.w: We are going to look into implementing the debdelta suite: http://debdelta.debian.net/ There will be a new blueprint for this.

(?)

Work Items