[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: When substitute download + decompression is CPU-bound
From: |
Pierre Neidhardt |
Subject: |
Re: When substitute download + decompression is CPU-bound |
Date: |
Fri, 15 Jan 2021 09:10:46 +0100 |
Ludovic Courtès <ludo@gnu.org> writes:
> One takeaway for me is that zstd decompression remains an order of
> magnitude faster than the others, regardless of the compression level.
>
> Another one is that at level 10 and higher zstd achieves compression
> ratios that are more in the ballpark of lzip.
Hmmm, this is roughly true for lzip < level 6, but as soon as lzip hits level 6
(the default!) it compresses up to twice as much!
> If we are to change the compression methods used at ci.guix.gnu.org, we
> could use zstd >= 10.
On Guillaume's graph, the compression speed at the default level 3 is
about 110 MB/s, while at level 10 it's about 40 MB/s, which is
approximately the gzip speed.
If server compression time does not matter, then I agree, level >= 10
would be a good option.
What about zstd level 19 then? It's as slow as lzip to compress, but
decompresses still blazingly fast, which is what we are trying to
achieve here, _while_ offering a compression ration in the ballpark of
lzip level 6 (but still not that of lzip level 9).
> We could also drop gzip, but there are probably pre-1.1 daemons out
> there that understand nothing but gzip¹, so perhaps that’ll have to
> wait. Now, compressing substitutes three times may be somewhat
> unreasonable.
Agreed, maybe release an announcement and give it a few months / 1 year?
Cheers!
--
Pierre Neidhardt
https://ambrevar.xyz/
signature.asc
Description: PGP signature
- Re: When substitute download + decompression is CPU-bound, Guillaume Le Vaillant, 2021/01/07
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2021/01/07
- Re: When substitute download + decompression is CPU-bound, Ludovic Courtès, 2021/01/14
- Re: When substitute download + decompression is CPU-bound, Nicolò Balzarotti, 2021/01/14
- Re: When substitute download + decompression is CPU-bound,
Pierre Neidhardt <=
- Re: When substitute download + decompression is CPU-bound, Ludovic Courtès, 2021/01/28
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, Guillaume Le Vaillant, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, Nicolò Balzarotti, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, Pierre Neidhardt, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, Guillaume Le Vaillant, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, Nicolò Balzarotti, 2021/01/29
- Re: When substitute download + decompression is CPU-bound, zimoun, 2021/01/29