AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
Torrent transmit12/29/2023 a Debian archive is a possibly very large repository of packages, including many different versions and architectures.The archive is too large to track efficiently as a single torrent: communication between the same users in many different torrents would be wasteful and hard to track to allow for fair downloading.a user would need to participate in possibly thousands of torrents simultaneously, wasting resources.There are too many packages to create individual torrents for each: if pieces are too small, the downloading becomes inefficient as a larger percentage of bandwidth is wasted on overhead.if multiple packages are present in a single piece, then a peer could waste a lot of bandwidth just to get the single package they need from the piece.The 2006 Google SoC project identified some of these concerns, and more have been added since (if you have a new problem, add it here).Īn archive is made up of a distribution of file sizes, many of which are smaller than the smallest piece size used by BitTorrent (currently 32 KB). ![]() These limitations of the current BitTorrent systems will mostly require modifications to the client and protocol to improve. Though the idea of implementing a BitTorrent-like solution to package distribution seems good, there are some problems with the current way that BitTorrent distributes files that make it unsuitable for a Debian archive. Certainly people have talked about ways of using gittorrent to achieve the same thing as this proposal suggests - but it's probably a good idea that both efforts carry on in parallel. Aria2 supports bittorrent too.Īnother derived BitTorrent protocol is gittorrent. It uses aria2 to download packages from multiple sources. There is another program called apt-metalink. apt-torrent is different in that it doesn't make any changes to the BitTorrent protocol, but rather it provides a wrapper around the BitTornado package which it then uses to download the package files. Unfortunately, the project failed due to time restrictions and the complexity of the project, but some good ideas came out of it.Ī similar project was accepted for the 2007 Google Summer of Code, and the resulting code can be found hereĪ program that is similar in idea to this one, apt-torrent, has been created by Arnaud Kyheng and is avaliable online. This was originally proposed and accepted as a Google Summer of Code project in 2006. This will further allow for future growth, both in the size of the archive and in the popularity of Debian. However, using a BitTorrent-like system, these voluntary mirrors could simply join the swarm of peers for the archive: mirroring only as much data and contributing only as much bandwidth as they can, providing multiple redundancies and being automatically load balanced by the system, and using the bandwidth savings to update their packages more frequently. Finally, non-primary mirrors may be slow to update to new versions of packages, and changes happen slowly as sources' lists must be updated manually by users. It also creates multiple central points of failure for users, as most depend on a single mirror, and does a poor job of evenly distributing the load, as some mirrors may be chosen more often by users. Some mirrors are already feeling the burden of the size, which has led to the introduction of partial mirroring. This system comes with some drawbacks though, especially as the size of the archive grows. ![]() These costs are currently being reduced by the voluntary mirroring system Debian uses to help distribute packages. Upon completion and widescale deployment of the service, the bandwidth and hardware costs of providing a very large Debian archive to hundreds of thousands of users will be dramatically reduced. The benefits of this project are clear, both to the Debian project and its mirrors, as well as any other developer wanting to host a popular archive but concerned with bandwidth costs. Proposed modifications to the user interface ![]() The original solutions proposed for the problems below Proposed protocol for interacting with APT Proposal for maintaining unique piece numbers This page is intended to collect information, ideas and comments related to adding BitTorrent functionality to the downloading of package files by Apt.
0 Comments
Read More
Leave a Reply. |