bug#15991: 24.3.50; Wishlist: Make test suite user-friendlier

[Eli Zaretskii]

This is a wishlist feature request: make the test suite easier to use
when some tests fail, and there's a need to investigate why and debug
them.

When you see any failures, the first thing you want to do is run only
one test that failed at a time.  But there's no way to do that, so it
seems, except by copying a (quite complex) command line from the
Makefile, while manually expanding every variable that's involved in
that.

Also, suppose one needs to run Emacs under a debugger for running a
test -- there doesn't seem to be a convenient way of doing that,
either.  Once again, start Emacs manually in GDB, then copy/paste the
command line there, and be sure to set the environment variables, like
EMACS_TEST_DIRECTORY before starting GDB.  Wait, and what if the test
forks, and you need to attach GDB to the correct Emacs process?

Another possible improvement would be for each test to automatically
save the log of its run, including any error messages, to a file.  As
things are now, one needs to do that manually as well, using shell
magic and/or 'tee', as the full log is extremely voluminous.

I wish all this could be easier, and I wish there would be a README in
that directory with instructions of how to do all that.

A good example of both the facilities and their README documentation
can be found in the Cairo distribution (look in the test/
subdirectory).

Thanks in advance.


In GNU Emacs 24.3.50.81 (i686-pc-mingw32)
 of 2013-11-28 on HOME-C4E4A596F7
Bzr revision: 115269 eliz@gnu.org-20131128194015-2eu4ceuanrqt0qwr
Windowing system distributor `Microsoft Corp.', version 5.1.2600
Configured using:
 `configure --prefix=/d/usr --enable-checking=yes,glyphs 'CFLAGS=-O0
 -gdwarf-2 -g3''

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o r t - e m <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns disp-table w32-win
w32-vars tool-bar dnd fontset image regexp-opt fringe tabulated-list
newcomment lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer nadvice loaddefs button faces cus-face macroexp
files text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
w32notify w32 multi-tty emacs)