[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Wget-dev] wget2 | Memory corruption (#284)

From: Tim Rühsen
Subject: Re: [Wget-dev] wget2 | Memory corruption (#284)
Date: Tue, 04 Jun 2019 07:42:14 +0000

@andrew-gitlab Your valgrind log shows normal wget behavior. Wget won't free 
used memory *explicitly* on exit (it's free'd anyways when exiting a process). 
But you *can* force explicit free'ing by using --debug.

Regarding excessive memory usage for complex sites... wget has to track *all* 
URLs it finds plus several other internal data for bookkeeping. That means the 
more it finds and has to download, the more memory it uses. Some sites are 
designed to generate dynamic URLs that keep changing with every request (most 
often the query part), so wget never comes to end end until... out of memory. 
Wget2 tries to counter measure this by keeping track of ETAGS that many servers 
use in a proper way (but not all).

Reply to this email directly or view it on GitLab: 
You're receiving this email because of your account on gitlab.com.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]