Re: [lmi] Making "dash -x" more atomic

[Greg Chicares]

On 2019-04-29 14:53, Vadim Zeitlin wrote:
> On Thu, 25 Apr 2019 15:02:06 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> On 2019-04-25 13:51, Vadim Zeitlin wrote:
> GC> > On Thu, 25 Apr 2019 12:39:06 +0000 Greg Chicares <address@hidden> wrote:
> GC> [...]
> GC> > GC>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=567648
[...]
>  Just an update: the good news is that there is a very simple (modifying
> one line and removing 2 other ones) patch fixing the problem in dash, see
> the end of this post:
> 
>       https://www.mail-archive.com/address@hidden/msg01880.html

Thanks.

> However there are 2 problems: the first one is that the patch hasn't been
> applied yet because of some (IMO misplaced) concerns about the inefficiency
> of having another buffer

I appreciate that they want dash to run scripts as quickly as possible
because that makes automated software builds faster. For that purpose,
though, '-x' wouldn't be used. AFAICS, this patch doesn't make dash's
"production" mode (without '-x') slower. It just makes this "debug"
mode more useful.

> and second one is that it turns out that even if
> the output consists of the same lines, the order of the lines can still
> very: and it does it not only in dash, but in bash too (I didn't mention it
> in my reply to dash mailing list, but zsh does produce consistent, albeit
> reversed compared to dash/bash, output even after hundreds of thousands of
> iterations).

Your correspondent on that list did say, though, that
| It takes hundreds or sometimes even thousands of runs
to see that particular inconsistency. AIUI, your patch would reduce
the combined total inconsistency by a couple orders of magnitude,
so I think it's a worthwhile improvement.

> GC> Logs that are identical clearly show no regression. Logs that are "almost"
> GC> identical require manual effort to compare.
> 
>  So there doesn't seem to be any way to generate perfectly reproducible
> xtrace output from a pipeline in neither dash nor bash. However the line
> order change is infrequent and can be dealt with by "git diff --no-index
> --color-moved=plain", so it shouldn't be too bad once some version of the
> patch above will be applied to dash and propagates to the next version of
> Debian (in the meanwhile, nothing prevents us from rebuilding Debian
> package with this patch ourselves and using it instead of the standard one,
> of course).

It's already not too bad. Using '--color-moved=plain' lets me
focus on the small percentage of diff hunks that are relevant:
a handful, rather than hundreds.

If we want reproducibility, we might use a different shell in
POSIX mode. That'd be easier than rebuilding dash ourselves.
But I don't think the effort would justify the benefits.