coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: new snapshot available: coreutils-9.2.18-ffd62.tar.xz


From: Pádraig Brady
Subject: Re: new snapshot available: coreutils-9.2.18-ffd62.tar.xz
Date: Thu, 13 Apr 2023 13:31:01 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:109.0) Gecko/20100101 Thunderbird/109.0

On 13/04/2023 07:05, Sam James wrote:

Pádraig Brady <P@draigBrady.com> writes:

On 12/04/2023 05:55, Sam James wrote:
Pádraig Brady <P@draigBrady.com> writes:

On 10/04/2023 22:52, Sam James wrote:
Sam James <sam@gentoo.org> writes:

Pádraig Brady <P@draigBrady.com> writes:

We plan to release coreutils-9.3 in the coming week
so any testing you can do on various different systems
between now and then would be most welcome.
This is a bug fix release coming about 3 weeks after the 9.2 release.

--------------------------------------

You can download the coreutils snapshot in xz format (5.7 MB) from:
       https://pixelbeat.org/cu/coreutils-ss.tar.xz

And verify with gpg or md5sum with:
       https://pixelbeat.org/cu/coreutils-ss.tar.xz.sig
       MD5 (coreutils-ss.tar.xz) = d208d306026fb42c128a787dffcba17a

--------------------------------------

To test follow this standard procedure:

       tar -xf coreutils-ss.tar.xz
       cd coreutils-9.2.18-ffd62/
       ./configure && make check VERBOSE=yes

Failures are reported, and details are in tests/test-suite.log
Please report/attach any issues to coreutils@gnu.org


I get one failure in tests/cp/backup-dir.sh inside our packaging:
I see now you've caught this as a non-portable shell construct &
fixed -
thanks! So ignore this one, and see the failures below instead.
```
+ env cp --version
[...]
FAIL tests/cp/backup-dir.sh (exit status: 1)
```
These I can naturally still hit:

In another environment (same machine but this time running manually,
outside of our packaging, and on a ZFS filesystem instead of tmpfs),
I get two failures:

1. tests/cp/sparse-2.sh.log: http://sprunge.us/jOCSEr (unfortunately I
can't consistently reproduce this one)

This might be due to slightly different I/O patterns between cp and dd.
The log above suggests that cp is not inducing holes while dd is.
It would be useful to get both strace files from running the following on that 
ZFS filesystem

    cd coreutils-9.2.18-ffd62

    printf x>k
    dd bs=1k seek=1 of=k count=255 < /dev/zero

    strace -e cp.strace src/cp --reflink=never --sparse=always k k2

    hole_size=$(stat -c %o k2)
    strace -o dd.strace src/dd if=k of=k2.dd bs=$hole_size conv=sparse

By the way, this time, I also got a tests/du/basic failure too:
http://sprunge.us/xEumX6.
For the commands you gave:
```
255+0 records in
255+0 records out
261120 bytes (261 kB, 255 KiB) copied, 0.00220926 s, 118 MB/s
2+0 records in
2+0 records out
262144 bytes (262 kB, 256 KiB) copied, 0.000550036 s, 477 MB/s
```
and:
cp.strace: http://sprunge.us/cng0Z9
dd.strace: http://sprunge.us/F0Vg2r
Note that I can't reproduce the original test failure every time,
only ~1/7, 1/8 times, so I might need to run these in a loop
to get you the output needed. Or these might be fine. Dunno. :)

The cp strace is interesting as it indicates the source file has holes detected,
which is unexpected. Perhaps zfs (with compression?) is detecting zeros
when SEEK_DATA/SEEK_HOLE are used? The interesting part of the strace is:

   lseek(3, 0, SEEK_DATA)                  = 0
   lseek(3, 0, SEEK_HOLE)                  = 131072
   read(3, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
   write(4, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
   lseek(3, 131072, SEEK_DATA)             = -1 ENXIO (No such device or 
address)
   ftruncate(4, 262144)                    = 0
   fallocate(4, FALLOC_FL_KEEP_SIZE|FALLOC_FL_PUNCH_HOLE, 131072, 131072) = 0

In any case the output pattern for dd is very similar:

   read(0, "x\0\0\0\0\0\0\0\0"..., 131072) = 131072
   write(1, "x\0\0\0\0\0\0\0\"..., 131072) = 131072
   read(0, "\0\0\0\0\0\0\0\0\"..., 131072) = 131072
   lseek(1, 131072, SEEK_CUR)              = 262144
   read(0, "", 131072)                     = 0
   lseek(1, 0, SEEK_CUR)                   = 262144
   ftruncate(1, 262144)                    = 0

Note ZFS had this issue with SEEK_HOLE but I don't think you're hitting that:
https://github.com/openzfs/zfs/issues/11900


I feature heavily in that bug, too :)

Given this is intermittent, I think this test may just be a little brittle,
by depending on file layout which is hard to match across all file systems.
The attached leverages our new --debug functionality to ensure
checking for zeros is enabled.


tests/cp/sparse-2.log: http://sprunge.us/2W8n6b.

I'll pursue this with the OpenZFS folks unless you think there's more we
can do here?

I don't think ZFS is doing anything wrong here,
just that it's a more complicated situation,
given that it's auto "sparseifying" the initial file k.

I see the issue with my test adjustment,
as it with ZFS it may indicate "SEEK_HOLE" is being used for sparse detection,
rather than the normal case of "zeros".
That's a bit of a limitation of the --debug implementation actually,
which I'll address with the first attached patch.
With both patches attached, the test should pass now.

2. ./tests/misc/tty-eof.log: http://sprunge.us/yWNByM

perl's expect module seems to be getting spurious exitstatus for some commands.
I suspect a race, and there is a hardcoded 10s in that script.
There are various other issues in that script actually.
Hopefully the attached addresses this.
Interestingly, it doesn't!
tests/misc/tty-eof.log: http://sprunge.us/UwqooK.
I just applied the patch (and verified the changes were there) and
ran 'make check' again rather than a rebuild, but I assume that
shouldn't matter given it's a test script change.

Drats. It's awkward to debug further when I can't repro.
Given it happens to random commands, I still suspect an issue with your Expect 
lib,
or with how we're using it, rather than an issue with the commands themselves.

No worries - I'll see if I can dig into it, I'm not optimistic though
because others with a similar environment can't seem to hit it either.

I'll let you know if I do get anywhere though.

cheers,
Pádraig

Attachment: 0001-copy-with-debug-indicate-if-NUL-detection-is-used-wi.patch
Description: Text Data

Attachment: 0002-tests-avoid-dependence-on-file-layout-for-cp-sparse-.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]