Re: test failures

[Stephen Berman]

On Fri, 21 Aug 2020 22:36:40 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> Are these known failures, or did something go wrong with my build, or
>> would perhaps make bootstrap eliminate the failures?
>
> I'm not getting any errors with the current trunk on Debian (with make
> check).

On Sat, 22 Aug 2020 09:31:29 +0300 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Date: Fri, 21 Aug 2020 22:23:05 +0200
>> 
>> 3 files contained unexpected results:
>>   lisp/progmodes/elisp-mode-tests.log
>>   lisp/help-fns-tests.log
>>   lisp/emacs-lisp/cl-generic-tests.log
>
> These all pass for me with the current master.

It seems that the failures arise because my Emacs sources are in a
directory on a different partition from my home directory, and the
source directory is symlinked from §HOME, and my Emacs build directory
is under $HOME.  When I do batch runs of the test three files in
question starting from my home directory (i.e. dereferencing the symlink
to the sources), I get the the failures I reported; but when I start the
batch runs from the real directory containing the test sources, the one
test in cl-generic-tests.el that failed now passes, and of the fourteen
xref tests in elisp-mode-tests.el that failed, now only five fail; yet
all five of the failing tests in help-fns-tests.el still fail.

However, concerning the latter, it seems that the tests expect
help-fns-function-description-header to return the quoted basename of
the file, while in my environment it returns the absolute filename.
Adjusting the regexp used in the tests fixes this issue (see the first
patch below) and then all tests pass regardless of where I start the
batch run.

Concerning cl-generic-tests.el, using file-truename as in the second
patch below prevents the one failure, also regardless of where I start
the batch run.

As for the failing xref tests in elisp-mode-tests.el, simply wrapping
the uses of expand-file-name in file-truename did not prevent the
errors.  I don't know cl-generic well enough to debug further without
assistence.

Steve Berman


diff --git a/test/lisp/help-fns-tests.el b/test/lisp/help-fns-tests.el
index da2b49e6b8..7782a41b9f 100644
--- a/test/lisp/help-fns-tests.el
+++ b/test/lisp/help-fns-tests.el
@@ -56,28 +56,28 @@ help-fns-test-interactive-built-in
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-macro ()
-  (let ((regexp "a Lisp macro in .subr\\.el")
+  (let ((regexp "a Lisp macro in .+subr\\.el")
         (result (help-fns-tests--describe-function 'when)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-defun ()
-  (let ((regexp "a compiled Lisp function in .subr\\.el")
+  (let ((regexp "a compiled Lisp function in .+subr\\.el")
         (result (help-fns-tests--describe-function 'last)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-lisp-defsubst ()
-  (let ((regexp "a compiled Lisp function in .subr\\.el")
+  (let ((regexp "a compiled Lisp function in .+subr\\.el")
         (result (help-fns-tests--describe-function 'posn-window)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-alias-to-defun ()
-  (let ((regexp "an alias for .set-file-modes. in .subr\\.el")
+  (let ((regexp "an alias for .set-file-modes. in .+subr\\.el")
         (result (help-fns-tests--describe-function 'chmod)))
     (should (string-match regexp result))))
 
 (ert-deftest help-fns-test-bug23887 ()
   "Test for https://debbugs.gnu.org/23887 ."
-  (let ((regexp "an alias for .re-search-forward. in .subr\\.el")
+  (let ((regexp "an alias for .re-search-forward. in .+subr\\.el")
         (result (help-fns-tests--describe-function 'search-forward-regexp)))
     (should (string-match regexp result))))
 

diff --git a/test/lisp/emacs-lisp/cl-generic-tests.el 
b/test/lisp/emacs-lisp/cl-generic-tests.el
index 5aa58782f3..9582907e51 100644
--- a/test/lisp/emacs-lisp/cl-generic-tests.el
+++ b/test/lisp/emacs-lisp/cl-generic-tests.el
@@ -240,7 +240,7 @@ cl-generic-tests--method-files--finds-methods
   (let ((retval (cl--generic-method-files 'cl-generic-tests--generic)))
     (should (equal (length retval) 2))
     (mapc (lambda (x)
-            (should (equal (car x) cl-generic-tests--this-file))
+            (should (equal (file-truename (car x)) 
cl-generic-tests--this-file))
             (should (equal (cadr x) 'cl-generic-tests--generic)))
           retval)
     (should-not (equal (nth 0 retval) (nth 1 retval)))))