[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] Multi-threaded getting of single file
From: |
Tim Ruehsen |
Subject: |
Re: [Bug-wget] Multi-threaded getting of single file |
Date: |
Wed, 10 Sep 2014 16:10:18 +0200 |
User-agent: |
KMail/4.14 (Linux/3.16-1-amd64; KDE/4.14.0; x86_64; ; ) |
On Wednesday 10 September 2014 18:26:45 Rajib Bandopadhyay wrote:
> Dear Developers,
>
> Is it possible to have multi-threaded
> downloads in wget to accelerate
> downloading?
Wget is not multi-threaded, but being worked on (git branch 'parallel-wget').
> Is user-controlled multi-threading of a
> single file (the file may be large)
> download possible in wget?
You could start Wget several times in parallel together with the (new) '--
start-pos' option. Together with --quota, you possibly could download chunks
of a file for later concatenation. Seems to be a hack and I did not test it.
Here is a snippet from the 'git log':
#####
commit dfa1f4e0646f6fab5b768fd67477601126070dbc
Author: Yousong Zhou <address@hidden>
Date: Wed Mar 19 23:42:04 2014 +0800
Make wget capable of starting downloads from a specified position.
This patch adds an option `--start-pos' for specifying starting position
of a HTTP or FTP download.
#####
If your server supports the 'metalink' format, missing or broken chunks could
be automatically detected and downloaded - again basically implemented in
parallel-wget.
> Is checking of the downloaded file
> possible with the server file using
> rsync like facility? If the file is
> corrupt is it possible to repair the
> damaged sections?
HTTP(S) nor FTP support this out-of-the-box.
You need a Metalink capable/configured server and client.
Metalink (http://www.metalinker.org/) seems not very widespread, though
Fedora, Ubuntu and SuSE seem to support it for distribution downloads.
Tim