bug#28506: coreutils 8.28 test suite hangs on APFS filesystem

[Jack Howarth]

On Mon, Sep 18, 2017 at 5:08 PM, Jim Meyering <address@hidden> wrote:

> On Mon, Sep 18, 2017 at 1:18 PM, Jack Howarth
> <address@hidden> wrote:
> > The coreutils 8.28 release, when built on macOS 10.13 under the new APFS
> > filesystem, produces a hang during the test suite run. The hang appears
> to
> > occur in the execution of coreutils-8.28/tests/split/filter.sh at..
> >
> > + yes
> > + head -n200K
> > + split -b1G '--filter=head -c1 >/dev/null'
> > + for mode in ''\'''\''' ''\''r/'\'''
> > + FILE = -
> >
> > according to the filter.log generated from executing the section of
> > split/filter.sh containing...
> >
> > yes | head -n200K | split -b1G --filter='head -c1 >/dev/null' || fail=1
> >
> > # Ensure that "endless" input is ignored when all filters finish
> > for mode in '' 'r/'; do
> >   FILE = '-'
> >   if test "$mode" = ''; then
> >     FILE = 'zero.in'
> >     truncate -s10T "$FILE" || continue
> >   fi
> >   for N in 1 2; do
> >     rm -f x??.n || framework_failure_
> >     timeout 10 sh -c \
> >       "yes | split --filter='head -c1 >\$FILE.n' -n $mode$N $FILE" ||
> fail=1
> >     # Also ensure we get appropriate output from each filter
> >     seq 1 $N | tr '0-9' 1 > stat.exp
> >     stat -c%s x??.n > stat.out || framework_failure_
> >     compare stat.exp stat.out || fail=1
> >   done
> > done
> >
> > I haven't opened a radar report yet as the Apple engineers can't look
> > directly at the source code for coreutils due to the GPLv3 licensing and
> > the test suite seems to be tangled up with the makefiles making it
> > impossible to extract a stand-alone test case reproducer to attach to a
> > radar bug report.
> >       Jack
> > ps Again, the hang seems to occur at the tail end of the log after it
> > emits...
> >
> > + FILE = -
> >
> > Any suggestions on how reduce this to a simpler test case? I would note
> > that the new APFS filesystem produces a failure in the python test
> suite...
> >
> > https://bugs.python.org/issue31380
> >
> > which is due to APFS not allowing files to be created with filenames that
> > contain unassigned codepoints in the Unicode 9.0 standard, whereas HFS+
> > does. So perhaps the coreutils hang might be a similar issue?
>
> Thank you for the testing and for the report.
>
> Is there any chance your failing test was via a python2 framework? I'm
> asking (on Pádraig's behalf) because there is a known problem whereby
> SIGPIPE is mishandled in that case, and that might explain this
> failure, since the data-generation phase relies on SIGPIPE killing
> this test's "yes" command.
>

I doubt it as the hang doesn't happen under 10.13 when run on a JHFS
formatted volume.
              Jack