groff 1.23.0.rc2 status report (was: [BUG] gropdf, tbl: Completely broken table)

[G. Branden Robinson]

Hi Alex,

At 2022-12-18T12:46:59+0100, Alejandro Colomar wrote:
> BTW, how is the status of 1.23.0?

Getting closer.  Details below.

> How many RC bugs are there?

Five, but one of these is basically "please release", so 4 of
development work and one of release engineering, which is partially in
Bertrand's hands.

Savannah's URLs are stupefyingly long, but I have an RC
[release-critical] bug dashboard for you.[1]

One of the development RC bugs is a documentation issue that I don't
plan to tackle until _after_ rc2 [release candidate 2] (unless Bertrand
is unavailable when I tell him I'm done hacking).  Because I will need
_something_ do while waiting for build and field test reports to come
back so I don't chew my fingers off.

So that's 3 issues.

Number one:

[mdoc] align rendering parameters with man(7)
https://savannah.gnu.org/bugs/index.php?62926

I noticed some inconsistencies between the way groff man(7) and groff
mdoc(7) format pages--ones that can't be attributed simply to the
different underlying macro languages.  To attack that problem I needed
to do something I had been putting off for 5 years, which is to learn
mdoc(7).  I think I now know enough mdoc(7) to write a man page of
average complexity in it without needing the package's own man page
open.  As I noted somewhere (in email or in an editorial comment at the
end of a commit message, maybe), groff mdoc(7) as it was in 1.22.4 "uses
too much Courier and too much bold".  I have significant esthetic
reservations about inline switches to Courier, particularly for syntax
synopses where the font falls back automatically to Times for
punctuation characters like brackets, but "variable" content is also in
Courier italic.  This combination makes the interword spacing wildly
inconsistent, and it can be hard to tell whether you're supposed to type
a space or not.  This is not helpful.

I plan to fiddle various groff mdoc(7) internal register settings so
that Courier is only used with ".Bd -literal" and ".Bd -literal".  These
are comparable to groff man(7)'s EX/EE regions.  Some more state
registers like the Booleans that identify whether you're "in" a "Name"
or "Synopsis" section, for instance, may be necessary.  But maybe not.
If I can keep the numerous strings that are used to customize the
styling of dozens of mdoc(7) macros under control by confining them to
switching font _styles_ rather than the family as well, this might be
low-effort.

Probably not a lot of this will be visible to terminal-only users.

A side effect of learning more about mdoc(7) is that I've fixed some
bugs in our implementation, including one that was documented in
4.3BSD-Reno.[2]

Number two:

[mdoc] handle rendering option strings and registers as man(7) does
https://savannah.gnu.org/bugs/index.php?63046

There is partial overlap with the previous bug.  It refers to the
registers and strings you can said to configure various rendering
parameters, long documented in groff man(7)'s "Options" section.

The idea is that when batch-rendering a collection of man(7) and mdoc(7)
pages, which didn't work in groff 1.22.4 or earlier, you want both macro
packages to honor the rendering parameters you specify.  groff mdoc(7)
has been falling farther out of sync with groff man(7) for years at this
point.  This one's almost done: all that is left is "HF", "MF", and "U".

Number three:

bug #62933: [man] produce hyperlinks in PDF output
https://savannah.gnu.org/bugs/index.php?62933

We've got man page cross-reference hyperlinks but, weirdly, not regular
URL support in groff_man(7).  To my dismay this doesn't appear to be
trivial.  Whoever came up with the "pdfhref" macro had Unix
command-line-itis, and constructed an interface that looked like that
instead of something you might use in a *roff document.  The problem is
that it expects the URL and, particularly, the link text to be macro
arguments.  This is a violent impedance mismatch with the way groff
man(7)'s email and web macros were set up to work; they format the link
text in a diversion.  Diversions and strings are not the same thing.
(Try putting diversion contents into a device control command and you
will rapidly be deafened by the screams of a thousand damned "can't
output node in transparent throughput" souls.)  So to solve this I have
to learn crazy stuff about PDFmark hotspots and how their boundaries are
computed, and then the thought of what happens when the link text
crosses a trap boundary (like a page break) causes the blood to drain
from my face.  At some point I may have to tell Regis I need to phone a
friend,[3] and I'll crawl to Deri the PDF wizard to knock some clues
into my head.

Then, though the bug doesn't say this, I have to do it all over again
for mdoc(7), because that doesn't have support for this, either.  On the
bright side, I have a much better grasp of mdoc(7)'s internals than I
used to, and pretty much everything to do with page location traps is
the same now anyway, thanks to my incessant meddling^W^W^Wthe problems I
had to solve to get batch-rendering working.

By the way, we're up to 158 automated test scripts and 398 bugs fixed
since groff 1.22.4.  I have another statistic on changed files at the
end of this message.

> Are they growing faster than they are eaten by birds?  :P

No.  They've been slowly but steadily dwindling for the last 2-3 months.

In the meantime I continue to fix documentation problems as I spot them,
and fix the occasional "bonus bug" when someone reports one I recently
introduced.  (John Gardner found one in my change to .Nm, the
4.3BSD-Reno bug fix mentioned above.)  Also, today I pushed a fix for
what was the oldest open bug against groff, a problem with mm(7)'s
memorandum type 5.  Werner filed that in 2008.

I've also started to revise the groff_mdoc(7) man page itself.

A possible project for after groff 1.23.0 final is to write an essay
about how mdoc(7) is over-hyped and the grass is not greener over there.
It has some nice properties, but some perverse ones too.  Some people on
this mailing list over the years have suggested that the groff project's
2014 Mission Statement commits us to promoting mdoc(7) over man(7), and
doing what we can to erase the horrid crapware that is man(7).  The
document says nothing of the sort.[4]

I am beginning to learn that passionate mdoc(7) proponents are not
reliable witnesses.

> BTW, I've seen in groff really bad cases of broken indentation where
> some lines use tabs, others use spaces, and others use a mix.  What's
> the coding style regarding tabs in groff source code?  I want to know
> in case I send a patch some day.

It varies from one part of the source tree to another.  :-/  Most of
James Clark's C++ is pretty consistent.  It's two-space "tab" stops with
literal tab characters replacing every 8 spaces.  Emacs local variables
and Vim modeline info is available in many files; I tend to add the
latter the first time I work on a file, because I get annoyed when my
editor doesn't do the right thing.

Keep in mind that the stuff in contrib/ often has its own conventions.
Even though a lot of that is no longer maintained by its contributor, I
seldom feel the urge to force a change in indentation style on anything.

Personally I'm more frequently annoyed by the inconsistency in _macro
package_ indentation conventions.

> > More idiomatic still, and well worth considering for the future, is
> > setting _all_ of these descriptions in text blocks.  This table
> > looks to me like it was laid out for an 80-column terminal with the
> > excessively long descriptions manually broken.  This looks
> > suboptimal when typeset and will look ridiculous on a wide terminal.
> 
> Yep.  Probably I'll do that; but (probably) not soon.

I see no reason to hurry.  Clearly few people read the suffixes(7) page
or they'd have said something about what a time capsule it is.

> But if the intro(*) page references _everything_, then it would be a
> huge page (there are thousands of pages in the Linux man-pages).

That's true, but consider the topic at issue--pieces of file names.
It's hard to crawl, let alone walk, on a Unix system without some grasp
of file manipulation.  This is the system that brought us the idea that
"everything is a file" (even if Plan 9 did better at delivering on the
promise).

There may not be much point linking to it today, but ping me to pick a
fight with suffixes(7) in six months.  :)

> Although, in the PDF, I'd like to have an index.  That might help.

Early editions of the Unix manual had permuted indexes.  I haven't yet
developed any practical experience with preparing indexes of any sort
for *roff documents.  That's something I'd like to remedy.  One thing I
don't like is the SunOS man(7) convention of littering the page source
with `IX` macro calls (behavior that pod2man faithfully replicates).
They don't format anything, so like 4.2BSD ms(7) section headings they
duplicate text someone has already typed, and nothing, or nearly
nothing, _reads_ these IX tags to do something useful with them.  They
thus simply become clutter.

Remember my DC/TG proposal I re-raised the other day?  Anything you'd
want to tag as having semantic value is a sure-fire good candidate for
indexing.

> > Unofficial patch attached.
> 
> Do you like the patch?

"Like" is a strong word.  It makes the page less weird.  I'll stand by
it, if that's what you mean.

> Should I apply it, or is it just a draft?

Go right ahead.  The real work is in auditing that table line by line.

I saw reference to some compression program called "yabba" and I
thought, "what the Hutt?"

Regards,
Branden

[1] 
https://savannah.gnu.org/bugs/index.php?go_report=Apply&group=groff&func=&set=custom&msort=0&report_id=225&advsrch=0&bug_id=&summary=&submitted_by=0&resolution_id=0&assigned_to=0&bug_group_id=0&status_id=1&severity=0&category_id=0&plan_release_id=103&history_search=0&history_field=0&history_event=modified&history_date_dayfd=19&history_date_monthfd=12&history_date_yearfd=2022&chunksz=50&spamscore=5&boxoptionwanted=1#options

[2] 
https://git.savannah.gnu.org/cgit/groff.git/commit/?id=0d85615c62d07b028b972dff38bfa7589dad8cf1
[3] obligatory obscure cultural reference
[4] https://www.gnu.org/software/groff/groff-mission-statement.html

For fun I thought I'd see what the diffstat between 1.22.4 and master
looked like, and filtered out files with fewer than 1,000 lines of diff.

 ChangeLog                                          | 18832 +++++++++++++++----
 ChangeLog.122                                      |  5331 ++++++
 NEWS                                               |  2302 ++-
 contrib/glilypond/glilypond.pl                     |  1296 +-
 contrib/groffer/groffer.1.man                      |  3822 ----
 contrib/groffer/main_subs.pl                       |  2132 ---
 contrib/mm/groff_mm.7.man                          |  2444 ++-
 contrib/mom/om.tmac                                |  2339 ++-
 contrib/rfc1345/rfc1345.tmac                       |  1705 ++
 doc/groff.texi                                     | 18782 +++++++++---------
 doc/meref.me                                       |  2241 ---
 doc/meref.me.in                                    |  2439 +++
 doc/ms.ms                                          |  4378 +++++
 m4/groff.m4                                        |  1397 +-
 man/groff.7.man                                    |  9025 +++++----
 man/groff_char.7.man                               |  2992 +--
 man/groff_diff.7.man                               |  5039 +++--
 man/groff_font.5.man                               |  1252 +-
 man/groff_tmac.5.man                               |  1575 +-
 man/roff.7.man                                     |  2921 ++-
 src/devices/grohtml/post-html.cpp                  |  1282 +-
 src/devices/gropdf/gropdf.1.man                    |  1008 +-
 src/devices/grops/grops.1.man                      |  1615 +-
 src/preproc/eqn/eqn.1.man                          |  1621 +-
 src/preproc/pic/pic.1.man                          |  1196 +-
 src/preproc/refer/refer.1.man                      |  1260 +-
 src/preproc/tbl/tbl.1.man                          |  2404 ++-
 src/roff/groff/groff.1.man                         |  2844 +--
 src/roff/grog/subs.pl                              |  1266 --
 src/roff/troff/input.cpp                           |  1424 +-
 src/roff/troff/troff.1.man                         |  1012 +-
 tmac/an.tmac                                       |  1553 +-
 tmac/{doc.tmac-u => doc.tmac}                      |  1218 +-
 tmac/groff_man.7.man                               |  2613 ---
 tmac/groff_man.7.man.in                            |  4187 +++++
 tmac/groff_mdoc.7.man                              |  3470 ++--
 tmac/groff_ms.7.man                                |  3508 ++--
 tmac/hyphenex.us                                   |  1454 --
 960 files changed, 126084 insertions(+), 72911 deletions(-)

I hope people will take me seriously when I suggest that groff's
documentation has undergone revision.

signature.asc
Description: PGP signature