bug#18405: bug#28311: 25.2; Improve syntax highlighting in bat-mode

[Anders Lindgren]

Hi!

Unfortunately, I can't comment on the actual error found, as I wasn't involved in the fix...

Anyway, I am positively glad that you have tests for font-lock rules. However, using htmlfontify for this doesn't seem like the best tool for this. 1) The output of htmlfontify may change in the future, in which case you will need to rewrite all your tests, 2) HTML is hard to read, and 3) The ert explainer doesn't really help you here. If example, in this case all it tells you is that the resulting html strings have different lengths. On other words, it may work for snippets like this, but it would not scale well.

A couple of years ago I wrote a tool, faceup (https://github.com/Lindydancer/faceup), specifically for testing highlighting in Emacs using ert. It uses a simple markup language, so the result is easier to read than when using HTML. The reference text can be used inline (like in your test) or stored in a corresponding ".faceup" file, which makes it possible to test files thousands of lines long. When a test fails, the ert explainer will tell you which lines contained errors, what that lines contained, and what they should contain, so it's much easier to find errors. I have successfully used this is several of my font-lock packages. In addition, I've accumulated a number of source files in various programming languages in a regression test suite for font-lock (https://github.com/Lindydancer/font-lock-regression-suite).

If you would like to modernise your font-lock test, I would be happy to contribute it to Emacs (as I suggested in bug#16063).

    -- Anders

On Fri, Sep 22, 2017 at 7:01 PM, Glenn Morris <rgm@gnu.org> wrote:

Running 5 tests (2017-09-22 09:57:54-0700)
   passed  1/5  bat-test-fill-paragraph-comment
hfy batch mode ( *temp*-973592:bat-mode)
Test bat-test-fontification-iter-var-1 backtrace:
  signal(ert-test-failed (((should (equal (bat-test-fontify "echo %%a\
  ert-fail(((should (equal (bat-test-fontify "echo %%a\necho %%~dp1\ne
  (if (unwind-protect (setq value-17 (apply fn-15 args-16)) (setq form
  (let (form-description-19) (if (unwind-protect (setq value-17 (apply
  (let ((value-17 'ert-form-evaluation-aborted-18)) (let (form-descrip
  (let* ((fn-15 (function equal)) (args-16 (condition-case err (let ((
  (closure (t) nil (let* ((fn-15 (function equal)) (args-16 (condition
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name bat-test-fontification-iter-var-1 :do
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit(nil)
  eval((ert-run-tests-batch-and-exit nil))
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/progmodes/bat-mode-t
  command-line()
  normal-top-level()
Test bat-test-fontification-iter-var-1 condition:
    (ert-test-failed
     ((should
       (equal
        (bat-test-fontify "echo %%a
echo %%~dp1
echo %%~$PATH:I")
        "<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>"))
      :form
      (equal
       #("<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>" 22 26
(face font-lock-builtin-face)
64 65
(face font-lock-variable-name-face)
95 99
(face font-lock-builtin-face)
137 141
(face font-lock-variable-name-face)
171 175
(face font-lock-builtin-face)
213 221
(face font-lock-variable-name-face))
       "<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>")
      :value nil :explanation
      (arrays-of-different-length 228 263
                                  #("<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~dp1</span>
<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">~$PATH:I</span>" 22 26
(face font-lock-builtin-face)
64 65
(face font-lock-variable-name-face)
95 99
(face font-lock-builtin-face)
137 141
(face font-lock-variable-name-face)
171 175
(face font-lock-builtin-face)
213 221
(face font-lock-variable-name-face))
                                  "<span class=\"builtin\">echo</span> %%<span class=\"variable-name\">a</span>
<span class=\"builtin\">echo</span> %%~dp<span class=\"variable-name\">1</span>
<span class=\"builtin\">echo</span> %%~$<span class=\"variable-name\">PATH</span>:<span class=\"variable-name\">I</span>" first-mismatch-at 109)))
   FAILED  2/5  bat-test-fontification-iter-var-1
hfy batch mode ( *temp*-454955:bat-mode)
   passed  3/5  bat-test-fontification-var-decl
hfy batch mode ( *temp*-493544:bat-mode)
   passed  4/5  bat-test-fontification-var-delayed-exp
hfy batch mode ( *temp*-832943:bat-mode)
   passed  5/5  bat-test-fontification-var-exp

Ran 5 tests, 4 results as expected, 1 unexpected (2017-09-22 09:57:54-0700)

1 unexpected results:
   FAILED  bat-test-fontification-iter-var-1