RE: [Gnu-arch-users] New feature at the mirror + request for help

[Parker, Ron]

> From: James Blackwell [mailto:address@hidden

> There is a p2p app that you may not be familiar with called 
> bittorrent.
> They have solved a similiar problem. The way that they handle it is
> that they have built in a centralization concept called that 
> they call a
> "tracker". These trackers, which are easy to set up by most 
> people, are 
> used to track files that are shared by the general purpose users. 
> 
> By making these trackers so simple to implement, they have hit upon a
> new concept -- decentralized centralization (!). Sure, there's
> superservers, but there's so many of them, that nobody is in control.
> 
> What bittorrent has to teach us is that the problem isn't centralized
> servers. The problem to avoid is having a centralized server 
> that is too

Okay, this is too weird.  After asking you about the bandwidth etc.
requirements of the supermirror, I got to thinking about bittorrent as it
might relate to arch, down the road, especially as the supermirror becomes
overloaded.  It is an interesting idea for a lot of different reasons.

(over-generalization ahead)

As I checked into bittorrent, I discovered that it is really designed for
large files with a fairly concentrated and intense demand period.  For it to
be efficient, there must be multiple downloads going on at the same time.  I
believe its reasonable minimum related to this is 5 simultaneous downloads.
Also it uses a chunk size of 256KiB making the reasonable minimum file size
1-1.5MiB.  This may work well for newly released archives or continuations,
but I don't believe it would be very efficient for changeset distribution.

One word of warning, those downloading a torrent must at the same time be
sharing the torrent.  This does not work for anyone behind an opaque
firewall and they rapidly become supplier starved because they are not able
to give back to the network.

I haven't had time to get back to investigating it and to think about
whether or not their might be a way to adapt what bittorrent does to the
arch environment.