[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] Help: Why wget wall clock time much higher than download
Re: [Bug-wget] Help: Why wget wall clock time much higher than download time?
Thu, 20 Jun 2019 09:55:38 +0200
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.7.1
On 6/17/19 10:32 PM, David Bodin wrote:
> *wget --page-requisites --span-hosts --convert-links --adjust-extension
> --execute robots=off --user-agent Mozilla
> This command above provides the following stats:
> Total wall clock time: 35s
> Downloaded: 248 files, 39M in 4.2s (9.36 MB/s)
> This website takes about 5 seconds to download and display all files on a
> hard refresh in the browser.
> Why is the wall clock time *significantly longer* than the download time
> and is there a way to make it faster?
First of all, --random-wait waits 0.5 to 1.5 seconds after each
downloaded page. Don't use it - there have been times when web servers
blocked fast clients, but that shouldn't be the case today.
Wget uses just one connection for downloading, no compression by
default, no http/2.
You can try Wget2 which uses as many parallel connections as you like,
uses compression by default and http/2 if possible. Depending on the
HTTP server, Wget2 is often 10x faster then Wget just with it's default
You find the latest Wget2 tarball at
Instructions how to build at
Description: OpenPGP digital signature