Re: Compressed man pages (was: Accessibility of man pages (was: Playground pager lsp(1)))

[Mingye Wang]

On Wed, Apr 12, 2023 at 10:24 PM Alejandro Colomar
<alx.manpages@gmail.com> wrote:
> $ sudo make install-man prefix=/opt/local/man/xz___ -j LINK_PAGES=symlink 
> Z=.xz  XZFLAGS=      | wc -l

Small nitpick here as Kerin's recommended pb=0 isn't actually used.
https://bugs.gentoo.org/169260#c19 (from Kerin) suggests that we might
get one-third more.

I'm having trouble getting the Makefile to behave on MSYS2, but it
does shrink a manual copy of man*/ totalling 7.2 M (probably because
`exit` and `nan` didn't get checked out by git -- case-insensitivity
issues) down to 2.8 M (both `du --apparent-size -sh`).

> .bz2, .lz, and .xz have similar times, and tuning the compression
> doesn't produce important changes in speed

Or size. This is to be expected, since man pages are really tiny
files, to the point that compressors can't see much context. [Zstd and
brotli each have a "dictionary mode" to deal with this, but (a) Zstd
dict-file requires an extra flag on decompress (b) nobody has brotli,
which has a default dictionary, installed.]

> .xz, but I don't see any advantage of using .xz).

Going for `xz -9` only unnecessarily inflates the dictionary size
beyond the file size and therefore the mem requirement. The dictionary
size at -0 is 256 KiB, already enough for almost every man page in
existence. (gz -9 is 32 KiB, if I recall correctly.)

> Conclussions:
>
> Any compression formats other than .gz are unreasonably slow.
> I'd say either use .gz, or plain text, or prepare to
> contribute code yourself to man-db to optimize for your favourite
> compression format.

For every compression format someone adds, there's going to be one
more optional dependency...

Cheers,
Mingye