[Qemu-devel] [RFC 00/10] Add kernel-doc support to our Sphinx setup

[Peter Maydell]

Hi; this is an RFC patchset which I'm sending out mostly to provide a
work-in-progress snapshot that might be useful for Gabriel's GSoC
project on API documentation generation. You can find the equivalent
git branch here:
https://git.linaro.org/people/peter.maydell/qemu-arm.git/log/?h=sphinx-docs
(git url/branch for cloning:
  https://git.linaro.org/people/peter.maydell/qemu-arm.git sphinx-docs )

The patchset starts by pulling in the kernel-doc script from the
kernel sources and wiring it up in our Sphinx config.  The version of
the script is just from head-of-kernel-git from when I was working on
this at the beginning of the year; we should probably check whether
we should update it to something newer and/or something from a more
specific kernel release.

The patchset then demonstrates the functionality with two documents:
a standalone file that just has API docs of the functions from
bitops.h, and an extension of the existing memory.rst doc to include
a "detailed API docs" section at the end with the generated docs from
memory.h.  In both cases to get the build to not error out we need to
fix up syntax issues in the header comments.  For the memory.h stuff
in particular some of the fixes are a bit ropy, and I also ended up
using a big "disable nitpick mode" hammer to shut up an error
(leaving some non-fatal warnings still generated).

Broadly speaking the earlier patches in the series are more
"solid" than the later ones which are a little more hacky.

Some specific notes:

(1) our Makefile rune runs sphinx-build with -n, which enables
"nitpick mode", where sphinx will complain about references that it
wanted to hyperlink but which it doesn't have anywhere te link them
to.  This is something that I think I got from the kernel's makefile
rules, and it's obviously nice for catching typos, but sphinx also
complains about references to "uint8_t" and other system types
because obviously there are no definitions of those in our headers. 
I have a patch in there which tries to whitelist these, but this may
not be the best approach. (Nitpick mode might also be awkward if we
want to generate docs for API A which uses some types from API B
which we have not yet started generating docs for.)

(2) kernel-doc assumes the kernel C coding style which always uses
explicit 'struct'.  This clashes with QEMU's style which usually
hides 'struct' behind typedefs, and seems to result in needing to
write 'struct' in the doc comment to get it recognised as being
documentation of the struct (see eg "memory.h: attempted kernel-doc
fixes") and also failure to recognize that "#MemoryRegion" is a
reference to the MemoryRegion struct type (resulting in errors about
references not being found).  I'm not sure exactly what to do here --
it may be some combination of making kernel-doc more flexible in
coping with structs that hide behind typedefs, and fixing up syntax
in our doc comments.

(3) There are definitely some other syntax issues that still need
fixing -- for instance if you look at the generated documentation for
memory_region_init_ram_nomigrate() the "Description" section is just
a sentence fragment, because the doc comment has been misparsed due
to the way the descriptive text following "function() - " is split
over multiple lines.  We would need to do a more thorough
proofreading check and fix up the syntax as needed.

(4) Some Sphinx warnings generated are legitimate -- it complains
correctly that we don't have doc comments covering all the fields in
some of the structs in memory.h, for instance.  In an ideal world we
would fix these too :-)

thanks
-- PMM

Peter Maydell (10):
  scripts/kernel-doc: Import kernel-doc script from Linux
  docs: Add kerneldoc sphinx module from Linux
  docs/conf.py: Enable use of kerneldoc sphinx extension
  docs/conf.py: Ignore some missing references in nitpick mode
  bitops.h: Silence kernel-doc complaints
  docs: Create bitops.rst as example of kernel-docs
  memory.h: Fix kerneldoc syntax issues
  docs: Add kernel-doc API documentation to memory.rst
  memory.h: attempted kernel-doc fixes
  Makefile: disable Sphinx nitpicking

 Makefile                 |    2 +-
 include/exec/memory.h    |   31 +-
 include/qemu/bitops.h    |   52 +-
 docs/conf.py             |   22 +-
 docs/devel/bitops.rst    |    8 +
 docs/devel/index.rst     |    1 +
 docs/devel/memory.rst    |    5 +
 docs/sphinx/kerneldoc.py |  150 +++
 scripts/kernel-doc       | 2223 ++++++++++++++++++++++++++++++++++++++
 9 files changed, 2458 insertions(+), 36 deletions(-)
 create mode 100644 docs/devel/bitops.rst
 create mode 100644 docs/sphinx/kerneldoc.py
 create mode 100755 scripts/kernel-doc

-- 
2.20.1