qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [PATCH] Acceptance test: add coverage tests for -smp op


From: Wainer dos Santos Moschetta
Subject: Re: [Qemu-devel] [PATCH] Acceptance test: add coverage tests for -smp option
Date: Mon, 19 Nov 2018 16:48:00 -0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2



On 11/12/2018 02:31 PM, Eduardo Habkost wrote:
On Fri, Nov 09, 2018 at 02:58:00PM -0500, Wainer dos Santos Moschetta wrote:
This adds tests for SMP option, by passing -smp with
various combinations of cpus, cores, threads, and sockets
values it checks that invalid topologies are not accepted
as well as missing values are correctly calculated.

Signed-off-by: Wainer dos Santos Moschetta<address@hidden>
The test code looks nice to me, but: how exactly do you expect
this to be executed?

'make check-acceptance' executes it with default parameters (i.e. cores=2, threads=2, sockets=2, and cpus=8).

It is possible to overwrite the default parameters by using Avocado's -p option. For example, 'avocado run -p sockets=1 -p cores=2 -p threads=1 tests/acceptance/smp_option_coverage.py'.

Do we have a test runner or multiplexer configuration already
able to generate the cores/threads/sockets/cpus parameters?

I did not have any variants file until you asked. Then I realized the problems (see inline below) that I will need to address on a v2 patch.

Anyway, the variants YAML file may look like this (adapted from an example created by Cleber Rosa):

# cat smp_variants.yaml
!mux
min:
  sockets: 1
  cores: 2
  threads: 1

medium:
  sockets: 2
  cores: 2
  threads: 2

max:
  sockets: 2
  cores: 8
  threads: 16

The smp_variants.yaml defines 3 variants (min, medium, and max), each with a different SMP topology. You can run the tests as:

# avocado run tests/acceptance/smp_option_coverage.py -m smp_variants.yaml
JOB ID     : 08d9736942e550226de9c3425a9b65c378b6654a
JOB LOG    : /root/avocado/job-results/job-2018-11-19T13.19-08d9736/job.log
 (01/60) tests/acceptance/smp_option_coverage.py:SmpOption.test_cpus_eq_maxcpus;min-7e5d: PASS (0.04 s)
< output omitted >
 (60/60) tests/acceptance/smp_option_coverage.py:SmpOption.test_cpus_gt_cores_threads_sockets;max-a8e5: PASS (0.02 s) RESULTS    : PASS 60 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 6.25 s

If you wish I can distribute that file with this patch too. I can use Avocado's test data file mechanism to run the variants as described in:
https://avocado-framework.readthedocs.io/en/65.0/WritingTests.html#accessing-test-data-files

As an alternative I can document all that on docs/devel/testing.rst. Whatever you prefer.

Thanks for the review!

- Wainer


---
  tests/acceptance/smp_option_coverage.py | 218 ++++++++++++++++++++++++
  1 file changed, 218 insertions(+)
  create mode 100644 tests/acceptance/smp_option_coverage.py

diff --git a/tests/acceptance/smp_option_coverage.py 
b/tests/acceptance/smp_option_coverage.py
new file mode 100644
index 0000000000..ab68d1a67d
--- /dev/null
+++ b/tests/acceptance/smp_option_coverage.py
@@ -0,0 +1,218 @@
+# QEMU -smp option coverage test.
+#
+# Copyright (c) 2018 Red Hat, Inc.
+#
+# Author:
+#  Wainer dos Santos Moschetta<address@hidden>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later.  See the COPYING file in the top-level directory.
+
+from functools import reduce
+from avocado.utils.process import run
+
+from avocado_qemu import Test
+
+
+class SmpOption(Test):
+    """
+    Launches QEMU with various cpus, cores, threads, sockets, and maxcpus
+    combination through -smp option, to check it does not accept invalid SMP
+    topologies as well as it is able to calculate correctly any missing values.
+
+    :avocado: enable
+    :avocado: tags=slow,coverage
+    """
+    def setUp(self):
+        super().setUp()
+        self.cores = self.params.get('cores', default=2)
+        self.threads = self.params.get('threads', default=2)
+        self.sockets = self.params.get('sockets', default=2)
+        self.cpus = self.params.get('cpus', default=8)

The self.cpus variable should not be a parameter but rather calculated (cores * threads * sockets). Also needs to type convert the return of self.params.get from string to integer.

+
+    def get_smp_topology(self):
+        """
+        Returns a dict with the id of cores, threads and sockets.
+        """
+        res = self.vm.qmp('query-hotpluggable-cpus')['return']
+        cpus = [x['props'] for x in res]
+
+        return reduce(lambda x, y: {'core-id': 
x['core-id'].union([y['core-id']]),
+                                    'thread-id': 
x['thread-id'].union([y['thread-id']]),
+                                    'socket-id': 
x['socket-id'].union([y['socket-id']])},
+                      cpus, {'core-id': set(), 'thread-id': set(), 
'socket-id': set()})
+
+    @staticmethod
+    def build_option(**kwargs):
+        """
+        Builds string for the -smp option.
+        Use cpus, cores, threads, sockets, maxcpus keys.
+        """
+        option_list = []
+        if kwargs.get('cpus', None) is not None:
+            option_list.append(str(kwargs.get('cpus')))
+        for key, val in kwargs.items():
+            if key == 'cpus':
+                continue
+            option_list.append('%s=%s' % (key, val))
+
+        return ",".join(option_list)
+
+    def launch_and_check(self, expect_cores=1, expect_threads=1,
+                         expect_sockets=1, **kwargs):
+        """
+        Launches VM and check its SMP topology was correctly set.
+        Use cpus, cores, threads, sockets, and maxcpus keys to specify the
+        topology.
+        """
+        option = self.build_option(**{key: val for key, val in kwargs.items()
+                                      if key in ['cpus', 'cores', 'threads',
+                                                 'sockets', 'maxcpus']})
+        self.vm.add_args('-smp', option)
+        self.vm.launch()
+        smp = self.get_smp_topology()
+        self.assertEqual(smp['core-id'], set(range(0, expect_cores)))
+        self.assertEqual(smp['thread-id'], set(range(0, expect_threads)))
+        self.assertEqual(smp['socket-id'], set(range(0, expect_sockets)))
+
+    def launch_and_check_fail(self, **kwargs):
+        """
+        Launches VM and check the process exits with expected error code, for
+        cases where the topology is expected not valid.
+        """
+        option = self.build_option(**kwargs)
+        res = run("%s -smp %s" % (self.qemu_bin, option), timeout=10,
+                  ignore_status=True)
+        self.assertNotEqual(res.exit_status, 0)
+
+    # Passing cpus and maxcpus only.
+    #
+
+    def test_cpus_eq_maxcpus(self):
+        self.launch_and_check(cpus=self.cpus,
+                              maxcpus=self.cpus,
+                              expect_sockets=self.cpus)
+
+    def test_cpus_lt_maxcpus(self):
+        maxcpus = self.cpus * 2
+        self.launch_and_check(cpus=self.cpus,
+                              maxcpus=maxcpus,
+                              expect_sockets=maxcpus)
+
+    def test_cpus_gt_maxcpus(self):
+        self.launch_and_check_fail(cpus=self.cpus,
+                                   maxcpus=self.cpus // 2)
+
+    # Passing a combination of cores, threads and sockets only.
+    #
+
+    def test_no_cores_no_threads_no_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              expect_sockets=self.cpus)
+
+    def test_no_cores_no_threads_sockets(self):
+        self.launch_and_check(sockets=self.sockets,
+                              expect_sockets=self.sockets)
+
+    def test_no_cores_threads_no_sockets(self):
+        self.launch_and_check(threads=self.threads,
+                              expect_threads=self.threads)
+
+    def test_no_cores_threads_sockets(self):
+        self.launch_and_check(threads=self.threads,
+                              sockets=self.sockets,
+                              expect_threads=self.threads,
+                              expect_sockets=self.sockets)
+
+    def test_cores_no_threads_no_sockets(self):
+        self.launch_and_check(cores=self.cores,
+                              expect_cores=self.cores)
+
+    def test_cores_no_threads_sockets(self):
+        self.launch_and_check(cores=self.cores,
+                              sockets=self.sockets,
+                              expect_cores=self.cores,
+                              expect_sockets=self.sockets)
+
+    def test_cores_threads_no_sockets(self):
+        self.launch_and_check(cores=self.cores,
+                              threads=self.threads,
+                              expect_cores=self.cores,
+                              expect_threads=self.threads)
+
+    def test_cores_threads_sockets(self):
+        self.launch_and_check(cores=self.cores,
+                              threads=self.threads,
+                              sockets=self.sockets,
+                              expect_cores=self.cores,
+                              expect_threads=self.threads,
+                              expect_sockets=self.sockets)
+
+    # Passing cpus always and a combination of cores, threads and sockets.
+    #
+
+    def test_cpus_no_cores_no_threads_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              sockets=self.sockets,
+                              expect_cores=self.cpus // self.sockets,
+                              expect_sockets=self.sockets)
+
+    def test_cpus_no_cores_threads_no_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              threads=self.threads,
+                              expect_threads=self.threads,
+                              expect_sockets=self.cpus // self.threads)
+
+    def test_cpus_no_cores_threads_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              threads=self.threads,
+                              sockets=self.sockets,
+                              expect_threads=self.threads,
+                              expect_sockets=self.sockets,
+                              expect_cores=self.cpus // (self.threads * 
self.sockets))
+
+    def test_cpus_cores_no_threads_no_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              cores=self.cores,
+                              expect_cores=self.cores,
+                              expect_sockets=self.cpus // self.cores)
+
+    def test_cpus_cores_no_threads_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              cores=self.cores,
+                              sockets=self.sockets,
+                              expect_cores=self.cores,
+                              expect_sockets=self.sockets,
+                              expect_threads=self.cpus // (self.cores * 
self.sockets))
+
+    def test_cpus_cores_threads_no_sockets(self):
+        self.launch_and_check(cpus=self.cpus,
+                              cores=self.cores,
+                              threads=self.threads,
+                              expect_cores=self.cores,
+                              expect_threads=self.threads,
+                              expect_sockets=self.cpus // (self.cores * 
self.threads))
+
+    def test_cpus_cores_threads_sockets(self):
+        self.launch_and_check(cpus=self.cores * self.threads * self.sockets,
+                              cores=self.cores,
+                              threads=self.threads,
+                              sockets=self.sockets,
+                              expect_cores=self.cores,
+                              expect_threads=self.threads,
+                              expect_sockets=self.sockets)
+
+    # Passing cpus less (or greater) than cores, threads and sockets.
+    #
+
+    def test_cpus_lt_cores_threads_sockets(self):
+        self.launch_and_check_fail(cpus=(self.cores * self.threads * 
self.sockets) // 2,
+                                   cores=self.cores,
+                                   threads=self.threads,
+                                   sockets=self.sockets,)
+
+    def test_cpus_gt_cores_threads_sockets(self):
+        self.launch_and_check_fail(cpus=self.cores * self.threads * 
self.sockets * 2,
+                                   cores=self.cores,
+                                   threads=self.threads,
+                                   sockets=self.sockets)
--
2.17.2




reply via email to

[Prev in Thread] Current Thread [Next in Thread]