Re: [Qemu-devel] Sphinx for QEMU docs? (and a doc-comment format question)

[Daniel P. Berrange]

On Sat, Nov 05, 2016 at 06:42:23PM +0000, Peter Maydell wrote:
> Hi; I had a play on Friday afternoon with Sphinx, which is the
> restructured-text documentation processor that the Linux kernel is
> just switching to for their documentation. My first impression is
> that it would probably be useful for us too (and if the kernel
> is using it this also suggests that it's a fairly reasonable
> tool to rely on that isn't going to stagnate or vanish on us).
> 
> In particular I think we could:
>  * set up a framework for our in-tree docs/ which gives us a
>    place to put new docs (both for-users and for-developers) --
>    I think having someplace to put things will reduce the barrier
>    to people writing useful new docs
>  * gradually convert the existing docs to rst
>  * use the sphinx extension features to pull in the doc-comments
>    we have been fairly consistently writing over the last few years
>    (for instance a converted version of docs/memory.txt could pull
>    in doc comments from memory.h; or we can just write simple
>    wrapper files like a "Bitmap operations" document that
>    displays the doc comments from bitops.h)
>  * eventually replace our current use of texinfo, since Sphinx
>    can output PDF and manpages as well as HTML
> 
> Because Sphinx only parses doc-comments in files when a .rst
> document in the tree specifically asks for them, this means we
> can do a gradual transition. (Tools like gtkdoc want to parse
> every file in the tree at once which gives a nasty big-bang
> requirement to fix a lot of doc comment syntax errors all at once.)

FWIW, gtk-doc doesn't have to be a big-big - you simply pass
it the list of file names you want it to process - it doesn't
recursively find all itself.

That said, I've done some experiments with gtk-doc and QEMU
a couple of weeks back and it didn't work out well. In particular
it has an peculiar hardcoded assumption about the way typedefs &
enums are declared, which is different from common practice in
QEMU. As a result, gtk-doc simply looses / discards a lot of
type info in QEMU source causing it to get very confused. Having
looke at the source, there's no way to workaround this bug, and
I don't think we want to change QEMU source to match gkt-docs'
preferred style. So IMHO that rules out GTK-DOC for QEMU.

I've also did a few experiments using Doxygen, and while it
managed to cope with consuming QEMU's source code fine, the
way it lays out the resulting HTML pages is just awful IMHO.

> This does raise the question though of what format our doc
> comments actually are. Because we haven't been running any
> kind of document tool on the code there's been no syntax checking,
> so I suspect what we have in practice is a bit of a mishmash.
> The original intent I think was that they were gtkdoc format,
> but we also have some kernel-doc format comments (where we've
> borrowed headers from the kernel, and then subsequently
> copied that style in other doc comments), and probably some
> which aren't any formal syntax at all.
> We have as for as I can see just 3 gtkdoc "SECTION:"
> markers for doc-comments which aren't just documenting a
> particular function, type or structure.

Yep, no matter which tool we use, we're going to need to clean
up existing API docs comments to be consistent.

> Since the doc-comment parsing is handled by a Sphinx extension
> (which is in Python), we could in theory pick any syntax we
> liked. However if we said "use kernel-doc" we would be able
> to just borrow the kernel's kernel-doc extension without
> having to do a load of hacking on it (instead we'd need to
> fix up doc comments when we added documentation files that
> read in those source files, but we will need to do some of
> that anyway to fix accidental syntax errors).

Yes, it'd be nice to not have to write our own extension
if possible.

> Does anybody have strong opinions on doc-comment formats?
> Anybody with experience with the kernel-doc format and
> views on any shortcomings?
> 
> Sphinx is packaged in Debian stable so hopefully for most
> developers it would not be "bleeding edge have to have a
> custom install of this" tooling.
> 
> For an idea of how the docs tend to come out, see
> https://readthedocs.org/ which collects formatted docs
> for a large number of Sphinx-using projects.
> There are a couple of LWN articles about Sphinx for the kernel
> if you want more detail: https://lwn.net/Articles/692704/
> 
> With a little luck I may be able to put something up
> on Monday as a sort of minimal-demonstration of how
> this would look in QEMU.

Sounds like a worthwhile idea to me overall.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://entangle-photo.org       -o-    http://search.cpan.org/~danberr/ :|