RE: [PULL 00/30] Next patches

[Xu, Ling1]

Hi, All,
      Very appreciated for your time on reviewing our patch.
      The second CI failure caused by our patch has been addressed. One simple 
way is moving "#endif" in qemu/tests/bench/xbzrle-bench.c from line 46 to line 
450.
We have submitted patch v7 to update this modification. Thanks for your time 
again.

Best Regards,
Ling
      

-----Original Message-----
From: Stefan Hajnoczi <stefanha@gmail.com> 
Sent: Wednesday, November 16, 2022 2:58 AM
To: Juan Quintela <quintela@redhat.com>; Xu, Ling1 <ling1.xu@intel.com>; Zhao, 
Zhou <zhou.zhao@intel.com>; Jin, Jun I <jun.i.jin@intel.com>
Cc: qemu-devel@nongnu.org; Michael Tokarev <mjt@tls.msk.ru>; Marc-André Lureau 
<marcandre.lureau@redhat.com>; David Hildenbrand <david@redhat.com>; Laurent 
Vivier <laurent@vivier.eu>; Paolo Bonzini <pbonzini@redhat.com>; Daniel P. 
Berrangé <berrange@redhat.com>; Peter Xu <peterx@redhat.com>; Stefan Hajnoczi 
<stefanha@redhat.com>; Dr. David Alan Gilbert <dgilbert@redhat.com>; Thomas 
Huth <thuth@redhat.com>; qemu-block@nongnu.org; qemu-trivial@nongnu.org; 
Philippe Mathieu-Daudé <philmd@linaro.org>; Fam Zheng <fam@euphon.net>
Subject: Re: [PULL 00/30] Next patches

On Tue, 15 Nov 2022 at 10:40, Juan Quintela <quintela@redhat.com> wrote:
>
> The following changes since commit 98f10f0e2613ba1ac2ad3f57a5174014f6dcb03d:
>
>   Merge tag 'pull-target-arm-20221114' of 
> https://git.linaro.org/people/pmaydell/qemu-arm into staging 
> (2022-11-14 13:31:17 -0500)
>
> are available in the Git repository at:
>
>   https://gitlab.com/juan.quintela/qemu.git tags/next-pull-request
>
> for you to fetch changes up to d896a7a40db13fc2d05828c94ddda2747530089c:
>
>   migration: Block migration comment or code is wrong (2022-11-15 
> 10:31:06 +0100)
>
> ----------------------------------------------------------------
> Migration PULL request (take 2)
>
> Hi
>
> This time properly signed.
>
> [take 1]
> It includes:
> - Leonardo fix for zero_copy flush
> - Fiona fix for return value of readv/writev
> - Peter Xu cleanups
> - Peter Xu preempt patches
> - Patches ready from zero page (me)
> - AVX2 support (ling)
> - fix for slow networking and reordering of first packets (manish)
>
> Please, apply.
>
> ----------------------------------------------------------------
>
> Fiona Ebner (1):
>   migration/channel-block: fix return value for
>     qio_channel_block_{readv,writev}
>
> Juan Quintela (5):
>   multifd: Create page_size fields into both MultiFD{Recv,Send}Params
>   multifd: Create page_count fields into both MultiFD{Recv,Send}Params
>   migration: Export ram_transferred_ram()
>   migration: Export ram_release_page()
>   migration: Block migration comment or code is wrong
>
> Leonardo Bras (1):
>   migration/multifd/zero-copy: Create helper function for flushing
>
> Peter Xu (20):
>   migration: Fix possible infinite loop of ram save process
>   migration: Fix race on qemu_file_shutdown()
>   migration: Disallow postcopy preempt to be used with compress
>   migration: Use non-atomic ops for clear log bitmap
>   migration: Disable multifd explicitly with compression
>   migration: Take bitmap mutex when completing ram migration
>   migration: Add postcopy_preempt_active()
>   migration: Cleanup xbzrle zero page cache update logic
>   migration: Trivial cleanup save_page_header() on same block check
>   migration: Remove RAMState.f references in compression code
>   migration: Yield bitmap_mutex properly when sending/sleeping
>   migration: Use atomic ops properly for page accountings
>   migration: Teach PSS about host page
>   migration: Introduce pss_channel
>   migration: Add pss_init()
>   migration: Make PageSearchStatus part of RAMState
>   migration: Move last_sent_block into PageSearchStatus
>   migration: Send requested page directly in rp-return thread
>   migration: Remove old preempt code around state maintainance
>   migration: Drop rs->f
>
> ling xu (2):
>   Update AVX512 support for xbzrle_encode_buffer
>   Unit test code and benchmark code

This commit causes the following CI failure:

cc -m64 -mcx16 -Ilibauthz.fa.p -I. -I.. -Iqapi -Itrace -Iui/shader
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
-fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g -isystem 
/builds/qemu-project/qemu/linux-headers -isystem linux-headers -iquote . 
-iquote /builds/qemu-project/qemu -iquote /builds/qemu-project/qemu/include 
-iquote
/builds/qemu-project/qemu/tcg/i386 -pthread -U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE 
-Wstrict-prototypes -Wredundant-decls -Wundef -Wwrite-strings 
-Wmissing-prototypes -fno-strict-aliasing -fno-common -fwrapv 
-Wold-style-declaration -Wold-style-definition -Wtype-limits -Wformat-security 
-Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body -Wnested-externs 
-Wendif-labels -Wexpansion-to-defined
-Wimplicit-fallthrough=2 -Wno-missing-include-dirs -Wno-shift-negative-value 
-Wno-psabi -fstack-protector-strong -fPIE -MD -MQ 
libauthz.fa.p/authz_simple.c.o -MF libauthz.fa.p/authz_simple.c.o.d -o 
libauthz.fa.p/authz_simple.c.o -c ../authz/simple.c In file included from 
../authz/simple.c:23:
../authz/trace.h:1:10: fatal error: trace/trace-authz.h: No such file or 
directory
1 | #include "trace/trace-authz.h"
| ^~~~~~~~~~~~~~~~~~~~~

https://gitlab.com/qemu-project/qemu/-/jobs/3326576115

I think the issue is that the test links against objects that aren't present 
when qemu-user only build is performed. That's my first guess, I might be wrong 
but it is definitely this commit that causes the failure (I bisected it).

There is a second CI failure here:

clang -m64 -mcx16 -Itests/bench/xbzrle-bench.p -Itests/bench -I../tests/bench 
-I. -Iqapi -Itrace -Iui -Iui/shader
-I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
-I/usr/include/sysprof-4 -flto -fcolor-diagnostics -Wall -Winvalid-pch -Werror 
-std=gnu11 -O2 -g -isystem /builds/qemu-project/qemu/linux-headers -isystem 
linux-headers -iquote . -iquote /builds/qemu-project/qemu -iquote 
/builds/qemu-project/qemu/include -iquote
/builds/qemu-project/qemu/tcg/i386 -pthread -D_GNU_SOURCE
-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes 
-Wredundant-decls -Wundef -Wwrite-strings -Wmissing-prototypes 
-fno-strict-aliasing -fno-common -fwrapv -Wold-style-definition -Wtype-limits 
-Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wempty-body 
-Wnested-externs -Wendif-labels -Wexpansion-to-defined 
-Wno-initializer-overrides -Wno-missing-include-dirs -Wno-shift-negative-value 
-Wno-string-plus-int -Wno-typedef-redefinition 
-Wno-tautological-type-limit-compare -Wno-psabi 
-Wno-gnu-variable-sized-type-not-at-end -fstack-protector-strong 
-fsanitize=safe-stack -fsanitize=cfi-icall 
-fsanitize-cfi-icall-generalize-pointers -fno-sanitize-trap=cfi-icall -fPIE -MD 
-MQ tests/bench/xbzrle-bench.p/xbzrle-bench.c.o -MF 
tests/bench/xbzrle-bench.p/xbzrle-bench.c.o.d -o 
tests/bench/xbzrle-bench.p/xbzrle-bench.c.o -c ../tests/bench/xbzrle-bench.c
../tests/bench/xbzrle-bench.c:84:15: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(buffer512, buffer512, XBZRLE_PAGE_SIZE, ^
../tests/bench/xbzrle-bench.c:84:15: note: did you mean 'xbzrle_encode_buffer'?
./../migration/xbzrle.h:17:5: note: 'xbzrle_encode_buffer' declared here int 
xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, ^
../tests/bench/xbzrle-bench.c:146:15: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, test512, XBZRLE_PAGE_SIZE, ^
../tests/bench/xbzrle-bench.c:205:15: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, ^
../tests/bench/xbzrle-bench.c:268:13: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
rc512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, ^
../tests/bench/xbzrle-bench.c:345:15: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, ^
../tests/bench/xbzrle-bench.c:414:15: error: implicit declaration of function 
'xbzrle_encode_buffer_avx512' is invalid in C99 
[-Werror,-Wimplicit-function-declaration]
dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, ^

https://gitlab.com/qemu-project/qemu/-/jobs/3326576144

>
> manish.mishra (1):
>   migration: check magic value for deciding the mapping of channels
>
>  meson.build                   |  16 +
>  include/exec/ram_addr.h       |  11 +-
>  include/exec/ramblock.h       |   3 +
>  include/io/channel.h          |  25 ++
>  include/qemu/bitmap.h         |   1 +
>  migration/migration.h         |   7 -
>  migration/multifd.h           |  10 +-
>  migration/postcopy-ram.h      |   2 +-
>  migration/ram.h               |  23 +
>  migration/xbzrle.h            |   4 +
>  io/channel-socket.c           |  27 ++
>  io/channel.c                  |  39 ++
>  migration/block.c             |   4 +-
>  migration/channel-block.c     |   6 +-
>  migration/migration.c         | 109 +++--
>  migration/multifd-zlib.c      |  14 +-
>  migration/multifd-zstd.c      |  12 +-
>  migration/multifd.c           |  69 +--
>  migration/postcopy-ram.c      |   5 +-
>  migration/qemu-file.c         |  27 +-
>  migration/ram.c               | 794 +++++++++++++++++-----------------
>  migration/xbzrle.c            | 124 ++++++
>  tests/bench/xbzrle-bench.c    | 465 ++++++++++++++++++++
>  tests/unit/test-xbzrle.c      |  39 +-
>  util/bitmap.c                 |  45 ++
>  meson_options.txt             |   2 +
>  scripts/meson-buildoptions.sh |  14 +-
>  tests/bench/meson.build       |   4 +
>  28 files changed, 1379 insertions(+), 522 deletions(-)  create mode 
> 100644 tests/bench/xbzrle-bench.c
>
> --
> 2.38.1
>
>