[Qemu-commits] [qemu/qemu] 291b61: multifd: We already account for this

qemu-commits
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 291b61: multifd: We already account for this

From:	Richard Henderson
Subject:	[Qemu-commits] [qemu/qemu] 291b61: multifd: We already account for this packet on the...
Date:	Sat, 29 Apr 2023 02:11:18 -0700
  Branch: refs/heads/staging
  Home:   https://github.com/qemu/qemu
  Commit: 291b612833789563f711285287e02c097643f1c0
      
https://github.com/qemu/qemu/commit/291b612833789563f711285287e02c097643f1c0
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/multifd.c

  Log Message:
  -----------
  multifd: We already account for this packet on the multifd thread

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>


  Commit: 3fb55f450449dd2f97276c2b788bedfaab7d2138
      
https://github.com/qemu/qemu/commit/3fb55f450449dd2f97276c2b788bedfaab7d2138
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/meson.build
    A migration/migration-stats.c
    A migration/migration-stats.h
    M migration/migration.c
    M migration/multifd.c
    M migration/ram.c
    M migration/ram.h
    M migration/savevm.c

  Log Message:
  -----------
  migration: Move ram_stats to its own file migration-stats.[ch]

There is already include/qemu/stats.h, so stats.h was a bad idea.
We want this file to not depend on anything else, we will move all the
migration counters/stats to this struct.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>


  Commit: fc6fcdd71deaf72e0759c5fe363d88e28445f2d7
      
https://github.com/qemu/qemu/commit/fc6fcdd71deaf72e0759c5fe363d88e28445f2d7
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/migration-stats.c
    M migration/migration-stats.h
    M migration/migration.c
    M migration/multifd.c
    M migration/ram.c
    M migration/savevm.c

  Log Message:
  -----------
  migration: Rename ram_counters to mig_stats

migration_stats is just too long, and it is going to have more than
ram counters in the near future.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>


  Commit: bb6a3af32e0fb67f1c7a2cc05979ba58d3f856f8
      
https://github.com/qemu/qemu/commit/bb6a3af32e0fb67f1c7a2cc05979ba58d3f856f8
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/migration-stats.c
    M migration/migration-stats.h

  Log Message:
  -----------
  migration: Rename RAMStats to MigrationAtomicStats

It is lousely based on MigrationStats, but that name is taken, so this
is the best one that I came with.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>

---

If you have any good suggestion for the name, I am all ears.


  Commit: 2585c1d11c300e5e0a4c0829963f09f6a8f71380
      
https://github.com/qemu/qemu/commit/2585c1d11c300e5e0a4c0829963f09f6a8f71380
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c
    M migration/ram.h
    M migration/rdma.c

  Log Message:
  -----------
  migration/rdma: Split the zero page case from acct_update_position

Now that we have atomic counters, we can do it on the place that we
need it, no need to do it inside ram.c.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>


  Commit: 02c1fa1693f9d4f1e312c3081a7a090f2f711422
      
https://github.com/qemu/qemu/commit/02c1fa1693f9d4f1e312c3081a7a090f2f711422
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c
    M migration/ram.h
    M migration/rdma.c

  Log Message:
  -----------
  migration/rdma: Unfold last user of acct_update_position()

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Lukas Straub <lukasstraub2@web.de>


  Commit: 32bcb96551ac23e7f9cf21d868d113ea7a0a9670
      
https://github.com/qemu/qemu/commit/32bcb96551ac23e7f9cf21d868d113ea7a0a9670
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/tls.c

  Log Message:
  -----------
  migration: Drop unused parameter for migration_tls_get_creds()

It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: c20062e46ef9e350c10b70317fb5ff31caec9a95
      
https://github.com/qemu/qemu/commit/c20062e46ef9e350c10b70317fb5ff31caec9a95
  Author: Juan Quintela <quintela@redhat.com>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/multifd.c
    M migration/postcopy-ram.c
    M migration/tls.c
    M migration/tls.h

  Log Message:
  -----------
  migration: Drop unused parameter for migration_tls_client_create()

It is not needed since we moved the accessor for tls properties to
options.c.

Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>


  Commit: 5d3b09409965beb372df2f9d723d8ccf5bde5fec
      
https://github.com/qemu/qemu/commit/5d3b09409965beb372df2f9d723d8ccf5bde5fec
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  qtest/migration-test.c: Add tests with compress enabled

There has never been tests for migration with compress enabled.

Add suitable tests, testing with compress-wait-thread = false
too.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 94c3c8df5e1e6cbc97cd53c91b1d35abe829db8a
      
https://github.com/qemu/qemu/commit/94c3c8df5e1e6cbc97cd53c91b1d35abe829db8a
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  qtest/migration-test.c: Add postcopy tests with compress enabled

Add postcopy tests with compress enabled to ensure nothing breaks
with the refactoring in the next commits.

preempt+compress is blocked, so no test needed for that case.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 280304a3164bbd1e032e38e83c662047f253ecc4
      
https://github.com/qemu/qemu/commit/280304a3164bbd1e032e38e83c662047f253ecc4
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Let the compress threads return a CompressResult enum

This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 7ab0eb0d01090c9e1576f26f3cb307a84758a71e
      
https://github.com/qemu/qemu/commit/7ab0eb0d01090c9e1576f26f3cb307a84758a71e
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Dont change param->block in the compress thread

Instead introduce a extra parameter to trigger the compress thread.
Now, when the compress thread is done, we know what RAMBlock and
offset it did compress.

This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 7fe37c4210cb59e4ea4d7da377fe83760ce4c09b
      
https://github.com/qemu/qemu/commit/7fe37c4210cb59e4ea4d7da377fe83760ce4c09b
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Reset result after sending queued data

And take the param->mutex lock for the whole section to ensure
thread-safety.
Now, it is explicitly clear if there is no queued data to send.
Before, this was handled by param->file stream being empty and thus
qemu_put_qemu_file() not sending anything.

This will be used in the next commits to move save_page_header()
out of compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: b4be6040c7deff7b55d7f613ec5c83757deebee7
      
https://github.com/qemu/qemu/commit/b4be6040c7deff7b55d7f613ec5c83757deebee7
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Do not call save_page_header() from compress threads

save_page_header() accesses several global variables, so calling it
from multiple threads is pretty ugly.

Instead, call save_page_header() before writing out the compressed
data from the compress buffer to the migration stream.

This also makes the core compress code more independend from ram.c.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 85f44f9c316c381120b21fdf9518e5d7b173cafd
      
https://github.com/qemu/qemu/commit/85f44f9c316c381120b21fdf9518e5d7b173cafd
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Call update_compress_thread_counts from compress_send_queued_data

This makes the core compress code more independend from ram.c.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 9b1efb40129d0583c9c73fc15b2c899ac817f508
      
https://github.com/qemu/qemu/commit/9b1efb40129d0583c9c73fc15b2c899ac817f508
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Remove last ram.c dependency from the core compress code

Make compression interfaces take send_queued_data() as an argument.
Remove save_page_use_compression() from flush_compressed_data().

This removes the last ram.c dependency from the core compress code.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 9e9bf8c21b68a881ebc47882b659905d9638a636
      
https://github.com/qemu/qemu/commit/9e9bf8c21b68a881ebc47882b659905d9638a636
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/meson.build
    A migration/ram-compress.c
    A migration/ram-compress.h
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Move core compression code into its own file

No functional changes intended.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 0c5190409d2e52b0a69007f7cfc3398cd0d352b9
      
https://github.com/qemu/qemu/commit/0c5190409d2e52b0a69007f7cfc3398cd0d352b9
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/ram-compress.c
    M migration/ram-compress.h
    M migration/ram.c

  Log Message:
  -----------
  ram.c: Move core decompression code into its own file

No functional changes intended.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 973704dded4b0730bfd8df4217f1bfa781f7a07e
      
https://github.com/qemu/qemu/commit/973704dded4b0730bfd8df4217f1bfa781f7a07e
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/qemu-file.c
    M migration/qemu-file.h
    M migration/ram-compress.c
    M migration/ram.c

  Log Message:
  -----------
  ram compress: Assert that the file buffer matches the result

Before this series, "nothing to send" was handled by the file buffer
being empty. Now it is tracked via param->result.

Assert that the file buffer state matches the result.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 5766ce817eac5b20c6fd65a145c818f954c0d935
      
https://github.com/qemu/qemu/commit/5766ce817eac5b20c6fd65a145c818f954c0d935
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/meson.build
    M migration/ram-compress.c

  Log Message:
  -----------
  ram-compress.c: Make target independent

Make ram-compress.c target independent.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: 05ecac612ec6a4bdb358e68554b4406ac2c8e25a
      
https://github.com/qemu/qemu/commit/05ecac612ec6a4bdb358e68554b4406ac2c8e25a
  Author: Lukas Straub <lukasstraub2@web.de>
  Date:   2023-04-28 (Fri, 28 Apr 2023)

  Changed paths:
    M migration/migration.c
    M migration/ram.c

  Log Message:
  -----------
  migration: Initialize and cleanup decompression in migration.c

This fixes compress with colo.

Signed-off-by: Lukas Straub <lukasstraub2@web.de>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>


  Commit: b93c11d4f113d2228651fea684f7048873e7c4ae
      
https://github.com/qemu/qemu/commit/b93c11d4f113d2228651fea684f7048873e7c4ae
  Author: Richard Henderson <richard.henderson@linaro.org>
  Date:   2023-04-29 (Sat, 29 Apr 2023)

  Changed paths:
    M migration/meson.build
    A migration/migration-stats.c
    A migration/migration-stats.h
    M migration/migration.c
    M migration/multifd.c
    M migration/postcopy-ram.c
    M migration/qemu-file.c
    M migration/qemu-file.h
    A migration/ram-compress.c
    A migration/ram-compress.h
    M migration/ram.c
    M migration/ram.h
    M migration/rdma.c
    M migration/savevm.c
    M migration/tls.c
    M migration/tls.h
    M tests/qtest/migration-test.c

  Log Message:
  -----------
  Merge tag 'migration-20230428-pull-request' of 
https://gitlab.com/juan.quintela/qemu into staging

Migration Pull request (20230429 vintage)

Hi

In this series:
- compression code cleanup (lukas)
  nice, nice, nice.
- drop useless parameters from migration_tls* (juan)
- first part of remove QEMUFileHooks series (juan)

Please apply.

# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmRMF90ACgkQ9IfvGFhy
# 1yPVUBAApRj2d6K6T0FwW8axL0LUD+7QXmMnTkhO9/QJDBHGban9hWuaH4n3Om+0
# 89tHl5Fo6bgOJMagUGfIuF1Xt9tJSFZMp3OMFxrV2+yhLqHQ7LlJSIFrO+ILeRQJ
# 3kKzTzBEb9LNVFPKrD2SyEvhGxxD1kHfnNydKdU3XqvyAWPXqr1BonxsF06+i5ud
# WRhEJ9iL9eFP4CkCPOMHxBhfme6gP+NkxaB6R2Eb30jNTP1U9nTlKaV2fGLVn/zY
# VRC1bV9Ujy3wGeKBp8Lm9FIa8yBIf/HImI+nWGFROYslECMAR0kMfEU+aWY5y8By
# t0xDD0OEh2QB7rjGpx1B3Pb2He5YznzQNG4kmjNqWcH+PWurRbBaN++1r1DHrQzk
# fhfsT3A9eWUM06JsMm2Y+IhHwzT+PddmjWjHDU/kPNn1+GnxFytC55Ie5ew62PLu
# jWtOTERSg38e6m2gdFM5PLnfP5aLmjH0An78CDFhVy8f8I0R89tIC915p4QECuaa
# IIetvv70NF92EOXEtV9IEt8XvCsOomXrUCCtQMSAthHsh37Vn3W2U8fmXluAclua
# AW/mllj1AqEaUHVvCyST5bq0xqzE90dqjUB474oKi+wtPIzpXb8LP0xVZvcU8zWo
# 53MNFybvcDoZdva1JynzUvug06k5o2t91wIkLyUvigZZQ+3Osrg=
# =rl9X
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 28 Apr 2023 08:00:45 PM BST
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [undefined]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* tag 'migration-20230428-pull-request' of 
https://gitlab.com/juan.quintela/qemu: (21 commits)
  migration: Initialize and cleanup decompression in migration.c
  ram-compress.c: Make target independent
  ram compress: Assert that the file buffer matches the result
  ram.c: Move core decompression code into its own file
  ram.c: Move core compression code into its own file
  ram.c: Remove last ram.c dependency from the core compress code
  ram.c: Call update_compress_thread_counts from compress_send_queued_data
  ram.c: Do not call save_page_header() from compress threads
  ram.c: Reset result after sending queued data
  ram.c: Dont change param->block in the compress thread
  ram.c: Let the compress threads return a CompressResult enum
  qtest/migration-test.c: Add postcopy tests with compress enabled
  qtest/migration-test.c: Add tests with compress enabled
  migration: Drop unused parameter for migration_tls_client_create()
  migration: Drop unused parameter for migration_tls_get_creds()
  migration/rdma: Unfold last user of acct_update_position()
  migration/rdma: Split the zero page case from acct_update_position
  migration: Rename RAMStats to MigrationAtomicStats
  migration: Rename ram_counters to mig_stats
  migration: Move ram_stats to its own file migration-stats.[ch]
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>


Compare: https://github.com/qemu/qemu/compare/9b112b1b79f0...b93c11d4f113
[Prev in Thread]
Current Thread
[Next in Thread]
[Qemu-commits] [qemu/qemu] 291b61: multifd: We already account for this packet on the..., Richard Henderson <=
Prev by Date: [Qemu-commits] [qemu/qemu] a36c86: qga/qapi-schema: Tidy up documentation of guest-fs...
Next by Date: [Qemu-commits] [qemu/qemu] 290afd: block/block-common: add zoned device structs
Previous by thread: [Qemu-commits] [qemu/qemu] 9b112b: Merge tag 'pull-qapi-2023-04-28' of https://repo.o...
Next by thread: [Qemu-commits] [qemu/qemu] 290afd: block/block-common: add zoned device structs
Index(es):
- Date
- Thread