Re: [RFC PATCH] tests/avocado: re-factor igb test to avoid timeouts

[Akihiko Odaki]

On 2023/03/22 20:22, Alex Bennée wrote:
> Akihiko Odaki <akihiko.odaki@daynix.com> writes:
>
> > On 2023/03/22 19:04, Alex Bennée wrote:
> > > Akihiko Odaki <akihiko.odaki@daynix.com> writes:
> > >
> > > > On 2023/03/22 3:17, Alex Bennée wrote:
> > > > > The core of the test was utilising "ethtool -t eth1 offline" to run
> > > > > through a test sequence. For reasons unknown the test hangs under some
> > > > > configurations of the build on centos8-stream. Fundamentally running
> > > > > the old fedora-31 cloud-init is just too much for something that is
> > > > > directed at testing one device. So we:
> > > > >      - replace fedora with a custom kernel + buildroot rootfs
> > > > >      - rename the test from IGB to NetDevEthtool
> > > > >      - re-factor the common code, add (currently skipped) tests for other
> > > > >         devices which support ethtool
> > > > >      - remove the KVM limitation as its fast enough to run in KVM or TCG
> > > >
> > > > I tried this but it seems the rootfs is corrupted:
> > > > 2023-03-22 13:53:06,728 __init__         L0153 DEBUG| EXT4-fs (sda):
> > > > INFO: recovery required on readonly filesystem
> > > > 2023-03-22 13:53:06,728 __init__         L0153 DEBUG| EXT4-fs (sda):
> > > > write access will be enabled during recovery
> > > > (snip)
> > > > 2023-03-22 13:54:24,534 __init__         L0153 DEBUG| EXT4-fs (sda):
> > > > I/O error while writing superblock
> > > > 2023-03-22 13:54:24,535 __init__         L0153 DEBUG| EXT4-fs (sda):
> > > > error loading journal
> > > > 2023-03-22 13:54:24,542 __init__         L0153 DEBUG| VFS: Cannot open
> > > > root device "sda" or unknown-block(8,0): error -5
> > > That's weird. I'm not seeing it when running here. However I can
> > > regenerate the whole thing and re-upload. Are there any other network
> > > tools worth adding?
> >
> > Only ethtool is needed for testing Intel NICs.
> >
> > > > I have a few more comments:
> > > >
> > > > - It may be possible to use microvm to trim it down further.
> > > Does microvm have PCI now? Most of the saving comes down to having a
> > > much lighter rootfs than the full cloud init of fedora. I think there is
> > > only really a syslogd and a klogd running at the start.
> >
> > microvm supports PCIe. You can enable it by specifying e.g., -M
> > microvm,pcie=on
> >
> > > > - I'm worried that having a rootfs for a single test is too costly to
> > > >     maintain. If you just want to avoid cloud-init, you can just
> > > >    specify:
> > > > init=/bin/sh
> > > Not really too bad. Buildroot makes it pretty easy. The config can
> > > be
> > > found here:
> > >     https://fileserver.linaro.org/s/Lk8z7kN3s3ds7kd
> >
> > Buildroot indeed automates everything to build rootfs, but it still
> > takes lots of time to build because it needs to build everything. It
> > also fetches sources from the origins of the packages if I understand
> > it correctly, and I'm worried that may harm the reproducibility of the
> > builds.
> >
> > These problems are not present with Fedora: you can add or replace a
> > particular component with a package (in this case ethtool is added),
> > and Fedora mirrors everything to build the binary.
>
> It's certainly preferable to lean on the distros and their
> infrastructure although:
>
>    - Fedora is a poor choice given the support lifetime

I don't think the lifetime matters that much as we do not care about the 
security of the VMs created during the tests.

>    - The various "full-fat" distros we run avocado tests for seem to be
>      very bloated (esp compared to my local Debian setup which boots very
>      quickly) >    - It's hard to argue with the time saving and stability 
> improvements,
>      especially as we are limited on CI time these days

You don't have to perform the normal userspace initialization the 
distribution comes with for this workload. We can just add something 
like the following to the kernel command line:
init=/bin/sh -- -c "rpm -i /path/to/ethtool.rpm && ethtool -t eth1 offline"

This is very similar to what Docker containers do. In Docker, you do not 
run daemons like systemd but directly run the application or use very 
small init.

The reason why I didn't write the tests in this way is because the 
current testing infrastructure does not have a means to provide a file 
to the guest. I think adding such functionality is much simpler than 
introducing Buildroot.

Regards,
Akihiko Odaki