[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

wget2 | Add option to "enforce" compression (#556)

From: @rockdaboot
Subject: wget2 | Add option to "enforce" compression (#556)
Date: Sun, 01 Aug 2021 10:46:24 +0000

Tim Rühsen created an issue: https://gitlab.com/gnuwget/wget2/-/issues/556

An option that makes wget2 stop downloading if the requested compression 
method(s) are ignored by the server.

HTTP doesn't know the "enforcing" of compression. The client just can inform 
the server what it supports. The server then can choose any of the offered 
compression types. The HTTP standard always implies "identity" (no compression) 
and has no way of saying "not identity".
That leaves the user in a situation where it is impossible to enforce 
Currently wget/wget2 offeer no way to avoid / error expensive uncompressed 

Add a new option --enforce-compression to avoid downloads if the server doesn't 
use one of the compression methods set via --compression=.

- won't really work with recursive downloads as short files are normally served 
uncompressed (compression may even make those files larger).
- if the file is already compressed (e.g. a tar.gz), it may be sent 
uncompressed as additional compression doesn't make sense.
- we have to think about Transport-Encoding: (badly supported by clients and 
- there are lot's of buggy servers out there
- should there be a content size threshold to not enforce compression for small 
files ?
- is this just a corner case so that adding/maintaining extra code doesn't make 
sense (see https://github.com/curl/curl/issues/7516#) ? We have to weigh 
complexity against use cases.

Reply to this email directly or view it on GitLab: 
You're receiving this email because of your account on gitlab.com.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]