[PATCH v6 00/21] Convert QAPI doc comments to generate rST instead of texinfo

[Peter Maydell]

This series switches all our QAPI doc comments over from texinfo
format to rST.  It then removes all the texinfo machinery, because
this was the last user of texinfo.

I think I have now resolved all of Markus' issues raised in his
review of v5, and IMHO this is ready to commit.  If there are still
things needing fixing, it would be nice if we were able to commit
patches 1-5, which are the ones which add the new indent-sensitive
code to the QAPI parser.  That would put a stop to the steady trickle
of doc-comment changes which break the new rules...

Also available as a git branch at
https://git.linaro.org/people/peter.maydell/qemu-arm.git sphinx-conversions-v6

The basic approach is somewhat similar to how we deal with kerneldoc
and hxtool: we have a custom Sphinx extension which is passed a
filename which is the json file it should run the QAPI parser over and
generate documentation for. Unlike 'kerneldoc' but somewhat like
hxtool, I have chosen to generate documentation by generating a tree
of docutils nodes, rather than by generating rST source that is then
fed to the rST parser to generate docutils nodes.  Individual lumps of
doc comment go to the rST parser, but the structured parts we render
directly. This makes it easier to get the structure and heading level
nesting correct.

Changes from v5:
 * rebased (in particular, updated to meson build system)
 * new patch 1 fixes indent issues that hit master since v5
 * new patch 2 makes block-latency-histogram-set's use of Example
   sections match everybody else's, instead of special casing it
   in the parser
 * the .gitignore got pruned after meson conversion so we only
   need to change git.orderfile now
 * slightly reordered patches to bring the parser.py indent change nearer
   the start of the series in the hopes of being able to get at least
   that much of the series into master
 * we now tell Sphinx about all the json files for dependency info,
   so editing a json file correctly rebuilds the docs
 * added a test case for the bad-de-indent parser error
 * Adopted the various Python scripting suggestions from Markus
 * We don't insist on section headings being the only thing in their
   doc comment block any more (the existing "must be first line"
   requirement is sufficient)
 * added a test case for doc-generation that does a compare of
   the sphinx plain-text builder output against a reference file
 * Added the Python source files for Sphinx extensions (including
   the QAPI source files) to the dependency lists for the manuals,
   so that changes made to them correctly trigger a docs rebuild
 * qemu-ga-ref.rst and qemu-qmp-ref.rst now have a TODO note about
   making the manual licensing more visible to readers
 * fixed bug in reported file/line info for some errors in rST
   in doc comments when using Sphinx 1.6
 * don't insist on section headers being in their own freeform doc
   comment block; they're (after commit dcdc07a97cbe) always the
   first line in the comment block, so just handle the possibility
   of having text after that.

There are a few things I have left out of this initial series:

 * unlike the texinfo, there is no generation of index entries
   or an index in the HTML docs
 * although there are HTML anchors on all the command/object/etc
   headings, they are not stable but just serial-number based
   tags like '#qapidoc-35', so not suitable for trying to link
   to from other parts of the docs

My view is that we can add niceties like this later; the series
already seems big enough to me.

thanks
-- PMM


Peter Maydell (21):
  qapi: Fix doc comment indentation again
  qapi/block.json: Add newline after "Example:" for
    block-latency-histogram-set
  tests/qapi/doc-good.json: Prepare for qapi-doc Sphinx extension
  scripts/qapi: Move doc-comment whitespace stripping to doc.py
  scripts/qapi/parser.py: improve doc comment indent handling
  qapi/machine.json: Escape a literal '*' in doc comment
  docs/sphinx: Add new qapi-doc Sphinx extension
  docs/interop: Convert qemu-ga-ref to rST
  docs/interop: Convert qemu-qmp-ref to rST
  qapi: Use rST markup for literal blocks
  qga/qapi-schema.json: Add some headings
  tests/qapi-schema: Convert doc-good.json to rST-style strong/emphasis
  meson.build: Move SPHINX_ARGS to top level meson.build file
  meson.build: Make manuals depend on source to Sphinx extensions
  tests/qapi-schema: Add test of the rST QAPI doc-comment outputn
  scripts/qapi: Remove texinfo generation support
  docs/devel/qapi-code-gen.txt: Update to new rST backend conventions
  scripts/texi2pod: Delete unused script
  Remove Texinfo related line from git.orderfile
  configure: Drop texinfo requirement
  Remove texinfo dependency from docker and CI configs

 docs/conf.py                               |   6 +-
 docs/devel/qapi-code-gen.txt               | 106 ++--
 docs/index.html.in                         |   2 -
 docs/interop/conf.py                       |   4 +
 docs/interop/index.rst                     |   2 +
 docs/interop/qemu-ga-ref.rst               |   9 +
 docs/interop/qemu-ga-ref.texi              |  80 ---
 docs/interop/qemu-qmp-ref.rst              |   9 +
 docs/interop/qemu-qmp-ref.texi             |  80 ---
 docs/meson.build                           |  11 +-
 docs/sphinx/qapidoc.py                     | 550 +++++++++++++++++++++
 tests/qapi-schema/doc-good.rst             |   5 +
 configure                                  |  12 +-
 meson.build                                | 103 +---
 qapi/block-core.json                       |  20 +-
 qapi/block.json                            |  12 +-
 qapi/machine.json                          |   2 +-
 qapi/migration.json                        | 102 ++--
 qapi/qapi-schema.json                      |   6 +-
 qga/qapi-schema.json                       |   8 +-
 .travis.yml                                |   1 -
 MAINTAINERS                                |   3 +-
 qapi/meson.build                           |   4 +-
 qga/meson.build                            |   3 +-
 scripts/checkpatch.pl                      |   2 +-
 scripts/coverity-scan/coverity-scan.docker |   1 -
 scripts/git.orderfile                      |   1 -
 scripts/qapi-gen.py                        |   2 -
 scripts/qapi/doc.py                        | 301 -----------
 scripts/qapi/gen.py                        |   7 -
 scripts/qapi/parser.py                     | 105 +++-
 scripts/texi2pod.pl                        | 536 --------------------
 tests/docker/dockerfiles/debian10.docker   |   1 -
 tests/docker/dockerfiles/fedora.docker     |   1 -
 tests/docker/dockerfiles/ubuntu.docker     |   1 -
 tests/docker/dockerfiles/ubuntu1804.docker |   1 -
 tests/docker/dockerfiles/ubuntu2004.docker |   1 -
 tests/qapi-schema/doc-bad-indent.err       |   1 +
 tests/qapi-schema/doc-bad-indent.json      |   8 +
 tests/qapi-schema/doc-bad-indent.out       |   0
 tests/qapi-schema/doc-good.json            |  27 +-
 tests/qapi-schema/doc-good.out             |  24 +-
 tests/qapi-schema/doc-good.texi            | 319 ------------
 tests/qapi-schema/doc-good.txt             | 288 +++++++++++
 tests/qapi-schema/meson.build              |  63 ++-
 45 files changed, 1223 insertions(+), 1607 deletions(-)
 create mode 100644 docs/interop/qemu-ga-ref.rst
 delete mode 100644 docs/interop/qemu-ga-ref.texi
 create mode 100644 docs/interop/qemu-qmp-ref.rst
 delete mode 100644 docs/interop/qemu-qmp-ref.texi
 create mode 100644 docs/sphinx/qapidoc.py
 create mode 100644 tests/qapi-schema/doc-good.rst
 delete mode 100644 scripts/qapi/doc.py
 delete mode 100755 scripts/texi2pod.pl
 create mode 100644 tests/qapi-schema/doc-bad-indent.err
 create mode 100644 tests/qapi-schema/doc-bad-indent.json
 create mode 100644 tests/qapi-schema/doc-bad-indent.out
 delete mode 100644 tests/qapi-schema/doc-good.texi
 create mode 100644 tests/qapi-schema/doc-good.txt

-- 
2.20.1