Re: colorful tests

[Bob Proulx]

Please CC me since I don't normally read automake-patches.  Thanks.

Ralf Wildenhues wrote:
> Does anybody know how to test that colorful output actually happens?

Since the colorful output of check.mk now uses 'tput' and 'tput' uses
TERM, it should be sufficient to force TERM and then test the output
has the expected escape sequences.

> I assume 'expect' or something similar can do this (I have no idea)?

Using 'expect' would certainly work but I think it is not needed for
this case.

I think a very simple test such as this should following example
should be fine.  I would convert all escape sequences into something
like hex through od just to avoid any complexity there.  I believe
this use of 'od' should be portable.  I am not sync'd up as to where
this should actually fit into automake and so this is simply an
example of how and not something ready use yet.

  fail=0

  expected="0000000 1b 5b 33 31 6d"
  testout=`TERM=ansi tput setaf 1 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

  expected="0000000 1b 5b 30 3b 31 30 6d"
  testout=`TERM=ansi tput sgr0 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

  expected="0000000"
  testout=`TERM=dumb tput setaf 1 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

  expected="0000000"
  testout=`TERM=dumb tput sgr0 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

  expected="0000000"
  testout=`TERM=vt100 tput setaf 1 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

  expected="0000000"
  testout=`TERM=vt100 tput sgr0 | od -tx1 | sed 1q`
  test _"$testout" != _"$expected" && fail=1

I think that testing those should be sufficient to have pretty good
confidence that things are working.

> Do I have to assume that tput may not work as expected if I redirect
> stdout to /dev/null as well?

No.  That should be okay.  A very normal use of tput is to have the
output redirected.  It is very typically captured into shell
variables.  So I think you are fine with regards to this.  Also I
think TERM is sufficient by itself.  I spot checked a few systems with
input redirected from /dev/null and they behaved the same using only
TERM for control.

> I'm not sure if the checks in am__tty_colors won't cause bogus
> output on stdout on some of the weirder systems.

It is sometimes hard to predict what will happen on weird systems but
with a small modification to the patch I think this should be okay.

> diff --git a/lib/Automake/tests/Makefile.in b/lib/Automake/tests/Makefile.in
> index f03165c..785c591 100644
> --- a/lib/Automake/tests/Makefile.in
> +++ b/lib/Automake/tests/Makefile.in
> @@ -48,6 +48,20 @@ mkinstalldirs = $(SHELL) $(top_srcdir)/lib/mkinstalldirs
>  CONFIG_CLEAN_FILES =
>  SOURCES =
>  DIST_SOURCES =
> +# If stdout is a non-dumb tty, use colors.  If test -t is not supported,
> +# then this fails; a conservative approach.  Of course do not redirect
> +# stdout here, just stderr...
> +am__tty_colors = \
> +red=; grn=; lgn=; blu=; std=;                                        \
> +test "X$$TERM" != Xdumb && test -t 1 2>/dev/null             \
> +&& (tput bold 1 && tput setaf 1 && tput sgr0) 2>/dev/null    \

I think that last line needs to be this line:

  +&& (tput bold 1 && tput setaf 1 && tput sgr0) >/dev/null 2>&1        \

Otherwise the commands actually output each of those escape sequences
to the terminal.  It _should_ be invisible since they are back to
back.  But that is where things might get strange.  I think that it is
better to avoid this.

Is it useful to optimize out that subshell created by (...)?  I think
it would be one less shell fork with this following:

  +&& tput bold 1 >/dev/null 2>&1                               \
  +&& tput setaf 1 >/dev/null 2>&1                              \
  +&& tput sgr0 >/dev/null 2>&1                                 \

Then this is repeated:

> diff --git a/tests/Makefile.in b/tests/Makefile.in

Is there a way to share this code between those two files?

Bob