Re: [PATCH] meson: Pass -j option to sphinx

[Fabiano Rosas]

Daniel P. Berrangé <berrange@redhat.com> writes:

> On Wed, Apr 26, 2023 at 01:03:46PM -0300, Fabiano Rosas wrote:
>> Save a bit of build time by passing the number of jobs option to
>> sphinx.
>> 
>> To avoid warnings from sphinx, alter our plugins to inform whether
>> they support parallelism. The two plugins touched are quite simple and
>> I don't see anything that would indicate they do not support being
>> called in parallel, so return True for both reads and writes.
>> 
>> before:
>>  $ time make -j16 man html
>>  ...
>>  [1/2] Generating docs/QEMU manual with a custom command
>>  [2/2] Generating docs/QEMU man pages with a custom command
>> 
>>  real    0m49.770s
>>  user    0m49.425s
>>  sys     0m0.716s
>> 
>> after:
>>  $ time make -j16 man html
>>  ...
>>  [1/2] Generating docs/QEMU manual with a custom command (wrapped by meson 
>> because command contains newlines)
>>  [2/2] Generating docs/QEMU man pages with a custom command (wrapped by 
>> meson because command contains newlines)
>>  real    0m30.153s
>>  user    1m5.945s
>>  sys     0m2.440s
>
> When I test this on Fedora 38, the docs build hangs forever.
>
> I ran sphinx directly and see it prints a traceback and then
> fails to exit after this error
>
> Traceback (most recent call last):
>   File "/usr/lib/python3.11/site-packages/sphinx/util/parallel.py", line 105, 
> in join
>     if not self._join_one():
>            ^^^^^^^^^^^^^^^^
>   File "/usr/lib/python3.11/site-packages/sphinx/util/parallel.py", line 129, 
> in _join_one
>     self._result_funcs.pop(tid)(self._args.pop(tid), result)
>   File "/usr/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 
> 478, in merge
>     self.env.merge_info_from(docs, env, self.app)
>   File "/usr/lib/python3.11/site-packages/sphinx/environment/__init__.py", 
> line 366, in merge_info_from
>     domain.merge_domaindata(docnames, other.domaindata[domainname])
>   File "/usr/lib/python3.11/site-packages/sphinx/domains/__init__.py", line 
> 295, in merge_domaindata
>     raise NotImplementedError('merge_domaindata must be implemented in %s '
> NotImplementedError: merge_domaindata must be implemented in <class 
> 'dbusdomain.DBusDomain'> to be able to do parallel builds!
>
> During handling of the above exception, another exception occurred:
>
> Traceback (most recent call last):
>   File "/usr/lib/python3.11/site-packages/sphinx/cmd/build.py", line 281, in 
> build_main
>     app.build(args.force_all, args.filenames)
>   File "/usr/lib/python3.11/site-packages/sphinx/application.py", line 347, 
> in build
>     self.builder.build_update()
>   File "/usr/lib/python3.11/site-packages/sphinx/builders/__init__.py", line 
> 310, in build_update
>     self.build(to_build,
>

Ah it seems the DBus plugin is only in effect when using sphinx > 4.
I'll try to reproduce and see what it takes to implement this
merge_domaindata. Thanks for testing.