[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-commits] [qemu/qemu] 34a680: migration/calc-dirty-rate: millisecon
|
From: |
Alex Bennée |
|
Subject: |
[Qemu-commits] [qemu/qemu] 34a680: migration/calc-dirty-rate: millisecond-granularity... |
|
Date: |
Tue, 10 Oct 2023 07:23:10 -0700 |
Branch: refs/heads/staging
Home: https://github.com/qemu/qemu
Commit: 34a68001f16208a574cbef0cd2d4e2d681faa604
https://github.com/qemu/qemu/commit/34a68001f16208a574cbef0cd2d4e2d681faa604
Author: Andrei Gudkov <gudkov.andrei@huawei.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M migration/dirtyrate.c
M migration/dirtyrate.h
M qapi/migration.json
Log Message:
-----------
migration/calc-dirty-rate: millisecond-granularity period
This patch allows to measure dirty page rate for
sub-second intervals of time. An optional argument is
introduced -- calc-time-unit. For example:
{"execute": "calc-dirty-rate", "arguments":
{"calc-time": 500, "calc-time-unit": "millisecond"} }
Millisecond granularity allows to make predictions whether
migration will succeed or not. To do this, calculate dirty
rate with calc-time set to max allowed downtime (e.g. 300ms),
convert measured rate into volume of dirtied memory,
and divide by network throughput. If the value is lower
than max allowed downtime, then migration will converge.
Measurement results for single thread randomly writing to
a 1/4/24GiB memory region:
+----------------+-----------------------------------------------+
| calc-time | dirty rate MiB/s |
| (milliseconds) +----------------+---------------+--------------+
| | theoretical | page-sampling | dirty-bitmap |
| | (at 3M wr/sec) | | |
+----------------+----------------+---------------+--------------+
| 1GiB |
+----------------+----------------+---------------+--------------+
| 100 | 6996 | 7100 | 3192 |
| 200 | 4606 | 4660 | 2655 |
| 300 | 3305 | 3280 | 2371 |
| 400 | 2534 | 2525 | 2154 |
| 500 | 2041 | 2044 | 1871 |
| 750 | 1365 | 1341 | 1358 |
| 1000 | 1024 | 1052 | 1025 |
| 1500 | 683 | 678 | 684 |
| 2000 | 512 | 507 | 513 |
+----------------+----------------+---------------+--------------+
| 4GiB |
+----------------+----------------+---------------+--------------+
| 100 | 10232 | 8880 | 4070 |
| 200 | 8954 | 8049 | 3195 |
| 300 | 7889 | 7193 | 2881 |
| 400 | 6996 | 6530 | 2700 |
| 500 | 6245 | 5772 | 2312 |
| 750 | 4829 | 4586 | 2465 |
| 1000 | 3865 | 3780 | 2178 |
| 1500 | 2694 | 2633 | 2004 |
| 2000 | 2041 | 2031 | 1789 |
+----------------+----------------+---------------+--------------+
| 24GiB |
+----------------+----------------+---------------+--------------+
| 100 | 11495 | 8640 | 5597 |
| 200 | 11226 | 8616 | 3527 |
| 300 | 10965 | 8386 | 2355 |
| 400 | 10713 | 8370 | 2179 |
| 500 | 10469 | 8196 | 2098 |
| 750 | 9890 | 7885 | 2556 |
| 1000 | 9354 | 7506 | 2084 |
| 1500 | 8397 | 6944 | 2075 |
| 2000 | 7574 | 6402 | 2062 |
+----------------+----------------+---------------+--------------+
Theoretical values are computed according to the following formula:
size * (1 - (1-(4096/size))^(time*wps)) / (time * 2^20),
where size is in bytes, time is in seconds, and wps is number of
writes per second.
Signed-off-by: Andrei Gudkov <gudkov.andrei@huawei.com>
Reviewed-by: Hyman Huang <yong.huang@smartx.com>
Message-Id:
<d802e6b8053eb60fbec1a784cf86f67d9528e0a8.1693895970.git.gudkov.andrei@huawei.com>
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
Commit: 320a6ccc769dc09ae7ad0b4838e322018f334bf4
https://github.com/qemu/qemu/commit/320a6ccc769dc09ae7ad0b4838e322018f334bf4
Author: Andrei Gudkov <gudkov.andrei@huawei.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M migration/dirtyrate.c
M qapi/migration.json
Log Message:
-----------
migration/dirtyrate: use QEMU_CLOCK_HOST to report start-time
Currently query-dirty-rate uses QEMU_CLOCK_REALTIME as
the source for start-time field. This translates to
clock_gettime(CLOCK_MONOTONIC), i.e. number of seconds
since host boot. This is not very useful. The only
reasonable use case of start-time I can imagine is to
check whether previously completed measurements are
too old or not. But this makes sense only if start-time
is reported as host wall-clock time.
This patch replaces source of start-time from
QEMU_CLOCK_REALTIME to QEMU_CLOCK_HOST.
Signed-off-by: Andrei Gudkov <gudkov.andrei@huawei.com>
Reviewed-by: Hyman Huang <yong.huang@smartx.com>
Message-Id:
<399861531e3b24a1ecea2ba453fb2c3d129fb03a.1693905328.git.gudkov.andrei@huawei.com>
Signed-off-by: Hyman Huang <yong.huang@smartx.com>
Commit: 7ff1b8c088ca958b4e55edf8e407375ab215f717
https://github.com/qemu/qemu/commit/7ff1b8c088ca958b4e55edf8e407375ab215f717
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M tests/qtest/bios-tables-test.c
Log Message:
-----------
tests/bios-tables-test: tcg-emulate opteron for mmio64 test
seabios starts to make the placement of the 64bit mmio window
depend on the physical address space. Run the testcase with
a fixed processor on tcg to avoid different results depending
on the host machine.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: 9a5ac9c261c374bd47bcb347ec1688b56eb954c2
https://github.com/qemu/qemu/commit/9a5ac9c261c374bd47bcb347ec1688b56eb954c2
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M tests/qtest/bios-tables-test-allowed-diff.h
Log Message:
-----------
tests/acpi: enable tests/data/acpi/q35/DSDT.mmio64 updates
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: 784155cdcb02ffaae44afecab93861070e7d652d
https://github.com/qemu/qemu/commit/784155cdcb02ffaae44afecab93861070e7d652d
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M roms/seabios
Log Message:
-----------
seabios: update submodule to git snapshot
git shortlog
------------
Gerd Hoffmann (7):
disable array bounds warning
better kvm detection
detect physical address space size
move 64bit pci window to end of address space
be less conservative with the 64bit pci io window
qemu: log reservations in fw_cfg e820 table
check for e820 conflict
José Martínez (1):
Fix high memory zone initialization in CSM mode
Lukas Stockner via SeaBIOS (1):
virtio-blk: Fix integer overflow for large max IO sizes
Mark Cave-Ayland (3):
esp-scsi: flush FIFO before sending SCSI command
esp-scsi: check for INTR_BS/INTR_FC instead of STAT_TC for command
completion
esp-scsi: handle non-DMA SCSI commands with no data phase
Niklas Cassel via SeaBIOS (1):
ahci: handle TFES irq correctly
Tony Titus via SeaBIOS (1):
Increase BUILD_MAX_E820 to 128
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: 14f5a7bae4cb5ca45a03e16b5bb0c5d766fd51b7
https://github.com/qemu/qemu/commit/14f5a7bae4cb5ca45a03e16b5bb0c5d766fd51b7
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M pc-bios/bios-256k.bin
M pc-bios/bios-microvm.bin
M pc-bios/bios.bin
M pc-bios/vgabios-ati.bin
M pc-bios/vgabios-bochs-display.bin
M pc-bios/vgabios-cirrus.bin
M pc-bios/vgabios-qxl.bin
M pc-bios/vgabios-ramfb.bin
M pc-bios/vgabios-stdvga.bin
M pc-bios/vgabios-virtio.bin
M pc-bios/vgabios-vmware.bin
M pc-bios/vgabios.bin
Log Message:
-----------
seabios: update binaries to git snapshot
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: c1774bdb11d01fad8c18633a5306c1721e719bac
https://github.com/qemu/qemu/commit/c1774bdb11d01fad8c18633a5306c1721e719bac
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M tests/data/acpi/q35/DSDT.mmio64
Log Message:
-----------
tests/acpi: update expected data files
DefinitionBlock ("", "DSDT", 1, "BOCHS ", "BXPC ", 0x00000001)
{
Scope (\)
{
OperationRegion (DBG, SystemIO, 0x0402, One)
Field (DBG, ByteAcc, NoLock, Preserve)
{
DBGB, 8
}
DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed,
NonCacheable, ReadWrite,
0x00000000, // Granularity
0x08000000, // Range Minimum
0xAFFFFFFF, // Range Maximum
0x00000000, // Translation Offset
0xA8000000, // Length
,, , AddressRangeMemory, TypeStatic)
DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed,
NonCacheable, ReadWrite,
0x00000000, // Granularity
0xC0000000, // Range Minimum
0xFEBFFFFF, // Range Maximum
0x00000000, // Translation Offset
0x3EC00000, // Length
,, , AddressRangeMemory, TypeStatic)
QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed,
Cacheable, ReadWrite,
0x0000000000000000, // Granularity
- 0x0000000200000000, // Range Minimum
- 0x00000009FFFFFFFF, // Range Maximum
+ 0x000000FF00000000, // Range Minimum
+ 0x00000106FFFFFFFF, // Range Maximum
0x0000000000000000, // Translation Offset
0x0000000800000000, // Length
,, , AddressRangeMemory, TypeStatic)
})
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: e83f3600f9ff7d9e0d014f328e64c49bb81b945c
https://github.com/qemu/qemu/commit/e83f3600f9ff7d9e0d014f328e64c49bb81b945c
Author: Gerd Hoffmann <kraxel@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M tests/qtest/bios-tables-test-allowed-diff.h
Log Message:
-----------
tests/acpi: disable tests/data/acpi/q35/DSDT.mmio64 updates
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit: 9a8981e68c8de5c13b4842d7cd169ba028c6ff2a
https://github.com/qemu/qemu/commit/9a8981e68c8de5c13b4842d7cd169ba028c6ff2a
Author: Stefan Hajnoczi <stefanha@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M migration/dirtyrate.c
M migration/dirtyrate.h
M qapi/migration.json
Log Message:
-----------
Merge tag 'dirtylimit-dirtyrate-pull-request-20231010' of
https://github.com/newfriday/qemu into staging
Dirtylimit and dirtyrate 20231010 patches PULL request
Dirty page rate measurement optimization.
Please apply, thanks, Yong.
# -----BEGIN PGP SIGNATURE-----
#
# iQJKBAABCAA0FiEEaF0CINwmSCgVLlfC3/Ij1rP+y5wFAmUklg0WHHlvbmcuaHVh
# bmdAc21hcnR4LmNvbQAKCRDf8iPWs/7LnIw6D/sECFML6dIDSckhKe1kRBT2oXRd
# lYz4/RRdxPJIJP0zS0yLYXd2d5vHzXdC3hETv7//QiWB2OP/UQnsZ70JCgF4DxIq
# bGL4BUHgaQmyUsyIQXceFznJJQOLs5DczDFJBR2zlQbu3YOAGeNJJmfxmVIEPfcL
# w5NK++g3nVZ3pLJBNblDBUwIW5uoOj6z85rCTc6pvddoTBcAqS0er1aTkuyx9jbB
# VsDdFNJumF6+VnE6QUITHX2knr3UmXc5dfXCJi4CLKRfx3nyK8vYydNawhPtobGD
# 0G5MZAPLO3JxdM67EccKj3I/kcAXU4iHu7rV5AscSI/rlfneGjfCup2Xd0we1GCR
# TD07AVDRuW+cS76nEtvDSRj6+8KarZEa3lVbvoPaXIazoHg3GjKylIMTAcGjFlzL
# AnGornOEZSfwNxT3BRvNHFdUNdA9ICZ90sEpWjeu80UNOT2JASOB6JE2VJBFnW81
# 4gaoIT74hpI8H2k/x3R8REPnw+SLMI+7VpcA2XcXuOQOdfk0+8zlvxPsJRBaKBzS
# d2es+CpUcmBxZdEQNOi905qxfCFLOhwcstJXyCvFQBp4f8l2SJfIE4liI29qpuma
# hubbOEo/EAVe8ywToHSYj2RU5hnj6gu0n3hvSeye76hS/K+bfvI+HZ3AX+rcRmP8
# vX6Vqs4wdNl5khEnNg==
# =ixu5
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 09 Oct 2023 20:08:45 EDT
# gpg: using RSA key 685D0220DC264828152E57C2DFF223D6B3FECB9C
# gpg: issuer "yong.huang@smartx.com"
# gpg: Good signature from "Yong Huang <yong.huang@smartx.com>" [unknown]
# gpg: WARNING: The key's User ID is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 685D 0220 DC26 4828 152E 57C2 DFF2 23D6 B3FE CB9C
* tag 'dirtylimit-dirtyrate-pull-request-20231010' of
https://github.com/newfriday/qemu:
migration/dirtyrate: use QEMU_CLOCK_HOST to report start-time
migration/calc-dirty-rate: millisecond-granularity period
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Commit: 0ad0d9dcd16f2ea816a413008ac5191ebef45c95
https://github.com/qemu/qemu/commit/0ad0d9dcd16f2ea816a413008ac5191ebef45c95
Author: Stefan Hajnoczi <stefanha@redhat.com>
Date: 2023-10-10 (Tue, 10 Oct 2023)
Changed paths:
M pc-bios/bios-256k.bin
M pc-bios/bios-microvm.bin
M pc-bios/bios.bin
M pc-bios/vgabios-ati.bin
M pc-bios/vgabios-bochs-display.bin
M pc-bios/vgabios-cirrus.bin
M pc-bios/vgabios-qxl.bin
M pc-bios/vgabios-ramfb.bin
M pc-bios/vgabios-stdvga.bin
M pc-bios/vgabios-virtio.bin
M pc-bios/vgabios-vmware.bin
M pc-bios/vgabios.bin
M roms/seabios
M tests/data/acpi/q35/DSDT.mmio64
M tests/qtest/bios-tables-test.c
Log Message:
-----------
Merge tag 'firmware/seabios-20231010-pull-request' of
https://gitlab.com/kraxel/qemu into staging
seabios: update to git snapshot
Give seabios a bit real world testing before tagging a release.
Update to release will follow later in the devel cycle.
v3: update mmio64 acpi test.
# -----BEGIN PGP SIGNATURE-----
#
# iQIzBAABCgAdFiEEoDKM/7k6F6eZAf59TLbY7tPocTgFAmUlNMUACgkQTLbY7tPo
# cTgp5RAA154WTGRGEoBveu8ocIVNzPxDB8esLBDmsQ7VSKo6e7wJvS+IyPBVYDSW
# O0p8Hr/2WlXV1BA1pVUyxzEN4AwqiRDBw4Elir6n//MF7r0wR84401hz6er+4O8j
# U/xkMXPL/mVV+dulNq+ACjt0nRWIQgdlxEnIl9AY7Nbx3l8NM70PrxRbiG/CtRUz
# gJW3uDU5xRWWGsI7tO8ZEuB2RRQ9B1uCw2ljx9QbxDonJABLUe6YVOI/QhBboX4B
# 2lIezGxRzDwvCdgHXCraakm0ZE6fL9FlHJ4tyapWt21ggkHA6+5oPcx/zisNuCIy
# EMiu6p7ttlC3AiaXUmjdCYboRzNKV+S6NvweqzQis5aUedomuFDt3yKo8BqHdzkO
# uxSFbzpVxXzSXer1OlRZfVGxME35oyfPfeRPVNpef4zg/XdtOeAhckMVHr72QqEd
# pRyR2x+CTxzJ2SsbQ21AS+0PUtEa3JHcna6VMlblCO4uRppb/cnpq6SA2m343PCB
# D9IVgtui7+VEbaTxMmkUfOLTP/+QQnF6KOCcOlGtB6HjzqNfJnwuu2k6RsYCkdZf
# 3tJIdVv72LsopZNHYrqm7iU3XpEBGqFdv3Dq3pm35uW1nRKBdr2VQPjANfttHKxd
# pUFve1EObHDRTbEFXI0ALvdlDLSX/54McTFF9BKuRPyuxJKyvPM=
# =7FXs
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 10 Oct 2023 07:25:57 EDT
# gpg: using RSA key A0328CFFB93A17A79901FE7D4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138
* tag 'firmware/seabios-20231010-pull-request' of
https://gitlab.com/kraxel/qemu:
tests/acpi: disable tests/data/acpi/q35/DSDT.mmio64 updates
tests/acpi: update expected data files
seabios: update binaries to git snapshot
seabios: update submodule to git snapshot
tests/acpi: enable tests/data/acpi/q35/DSDT.mmio64 updates
tests/bios-tables-test: tcg-emulate opteron for mmio64 test
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Compare: https://github.com/qemu/qemu/compare/cea3ea670fe2...0ad0d9dcd16f
- [Qemu-commits] [qemu/qemu] 34a680: migration/calc-dirty-rate: millisecond-granularity...,
Alex Bennée <=