Re: [Gnu-arch-users] review/merge request: sftp get/put-file() pipe-lining

[Aaron Bentley]

Scott Parish wrote:
> address@hidden/tla--devo--1.3--patch-1
> http://software.srparish.net/archives/address@hidden/
>
> While researching something else i was struck by the realization that
> file-get/put() for sftp should be pipe-lined, but i had neglected to
> realize and do such earlier.

> There is some room for debate on tuning the number of requests to
> pipeline(N): with get() its anticipated that there will be N-1 requests
> that will return EOF, so if we are get()ing multiple tiny files this is
> pure overhead. On the other hand, if there is a large lag, keeping N
> sufficiently large should minimize effects of lag.

Neat to hear!  I've been working on optimizing the revision-building 
process.  The first part is the backbuilder, which is settling down now, 
and I hope to get merged in the next month.  Then I'd like to optimize 
archive grovelling and patch retrieval for non-serial 
(pipelined/paralel)  pfses.

That could improve the download time on the hackerlab even more, by 
retrieving all the required patches as efficiently as it would retrieve 
one big file of equal size, and by making it much faster to construct a 
patch list in the first place.

Aaron