bug#58473: 29.0.50; todo-test-add-and-delete-file is a flaky test

[Stefan Kangas]

Matt Armstrong <matt@rfc20.org> writes:

> It seems that todo-test-add-and-delete-file is a flaky test.  It passes
> most of the time but sometimes not.

Are you still seeing this?  I can't remember having seen it, myself.

>
> Example error log:
>
> Running 39 tests (2022-10-12 09:37:27-0700, selector `(not (or (tag 
> :expensive-test) (tag :unstable) (tag :nativecomp)))')
> Test todo-test-add-and-delete-file backtrace:
>   signal(ert-test-failed (((should (equal todo-current-todo-file todo-
>   ert-fail(((should (equal todo-current-todo-file todo-test-file-1)) :
>   #f(compiled-function () #<bytecode -0x719631cd833b93d>)()
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name todo-test-add-and-delete-file :docume
>   ert-run-or-rerun-test(#s(ert--stats :selector ... :tests ... :test-m
>   ert-run-tests((not (or (tag :expensive-test) (tag :unstable) (tag :n
>   ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable) (
>   ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
>   eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
>   command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/calendar/todo-mode-t
>   command-line()
>   normal-top-level()
> Test todo-test-add-and-delete-file condition:
>     (ert-test-failed
>      ((should
>        (equal todo-current-todo-file todo-test-file-1))
>       :form
>       (equal nil 
> "/home/matt/git/emacs-noverlay/test/lisp/calendar/todo-mode-resources/todo-test-1.todo")
>       :value nil :explanation
>       (different-types nil 
> "/home/matt/git/emacs-noverlay/test/lisp/calendar/todo-mode-resources/todo-test-1.todo")))
>    FAILED   1/39  todo-test-add-and-delete-file (0.033038 sec) at 
> lisp/calendar/todo-mode-tests.el:823
> Type C-x C-q to return to Todo mode.
>
> Warning (todo):
> Type C-x C-q to return to Todo mode.
>
> This also runs a file format check and signals an error if
> the format has become invalid.  However, this check cannot
> tell if the number of items or categories changed, which
> could result in the file containing inconsistent information.
> You can repair this inconsistency by invoking the command
> `todo-repair-categories-sexp', but this will revert any
> renumbering of the categories you have made, so you will
> have to renumber them again (see `(todo-mode) Reordering
> Categories').
>
> Warning (todo):
> Type C-x C-q to return to Todo Archive mode.
>
> This also runs a file format check and signals an error if
> the format has become invalid.  However, this check cannot
> tell if the number of items or categories changed, which
> could result in the file containing inconsistent information.
> You can repair this inconsistency by invoking the command
> `todo-repair-categories-sexp', but this will revert any
> renumbering of the categories you have made, so you will
> have to renumber them again (see `(todo-mode) Reordering
> Categories').
>
>    passed   2/39  todo-test-current-file-in-edit-mode (0.270833 sec)
>    passed   3/39  todo-test-done-items-separator01-bol (0.129566 sec)
>    passed   4/39  todo-test-done-items-separator01-eol (0.145479 sec)
>    passed   5/39  todo-test-done-items-separator02-bol (0.111079 sec)
>    passed   6/39  todo-test-done-items-separator02-eol (0.112115 sec)
>    passed   7/39  todo-test-done-items-separator03-bol (0.156204 sec)
>    passed   8/39  todo-test-done-items-separator03-eol (0.154852 sec)
>    passed   9/39  todo-test-done-items-separator04-bol (0.171032 sec)
>    passed  10/39  todo-test-done-items-separator04-eol (0.193185 sec)
>    passed  11/39  todo-test-done-items-separator05-bol (0.154420 sec)
>    passed  12/39  todo-test-done-items-separator05-eol (0.152174 sec)
>    passed  13/39  todo-test-done-items-separator06-bol (0.169586 sec)
>    passed  14/39  todo-test-done-items-separator06-eol (0.126120 sec)
>    passed  15/39  todo-test-done-items-separator07 (0.187008 sec)
>    passed  16/39  todo-test-edit-item-date-month (0.192969 sec)
> Warning (todo):
> Type C-x C-q to return to Todo mode.
>
> This also runs a file format check and signals an error if
> the format has become invalid.  However, this check cannot
> tell if the number of items or categories changed, which
> could result in the file containing inconsistent information.
> You can repair this inconsistency by invoking the command
> `todo-repair-categories-sexp', but this will revert any
> renumbering of the categories you have made, so you will
> have to renumber them again (see `(todo-mode) Reordering
> Categories').
>
> Warning (todo):
> Type C-x C-q to return to Todo Archive mode.
>
> This also runs a file format check and signals an error if
> the format has become invalid.  However, this check cannot
> tell if the number of items or categories changed, which
> could result in the file containing inconsistent information.
> You can repair this inconsistency by invoking the command
> `todo-repair-categories-sexp', but this will revert any
> renumbering of the categories you have made, so you will
> have to renumber them again (see `(todo-mode) Reordering
> Categories').
>
>    passed  17/39  todo-test-edit-quit (0.354015 sec)
>    passed  18/39  todo-test-item-highlighting (0.030907 sec)
>    passed  19/39  todo-test-move-item01 (0.179885 sec)
>    passed  20/39  todo-test-move-item02 (0.135768 sec)
>    passed  21/39  todo-test-move-item03 (0.081981 sec)
>    passed  22/39  todo-test-move-item04 (0.087317 sec)
>    passed  23/39  todo-test-move-item05 (0.135557 sec)
>    passed  24/39  todo-test-multiline-item-indentation-1 (0.122302 sec)
> Type C-x C-q to return to Todo mode.
>
>    passed  25/39  todo-test-multiline-item-indentation-2 (0.166801 sec)
> Warning (todo):
> Type C-x C-q to return to Todo mode.
>
> This also runs a file format check and signals an error if
> the format has become invalid.  However, this check cannot
> tell if the number of items or categories changed, which
> could result in the file containing inconsistent information.
> You can repair this inconsistency by invoking the command
> `todo-repair-categories-sexp', but this will revert any
> renumbering of the categories you have made, so you will
> have to renumber them again (see `(todo-mode) Reordering
> Categories').
>
>    passed  26/39  todo-test-multiline-item-indentation-3 (0.316444 sec)
>    passed  27/39  todo-test-raise-lower-priority (0.122647 sec)
>    passed  28/39  todo-test-revert-buffer01 (0.134979 sec)
>    passed  29/39  todo-test-revert-buffer02 (0.120267 sec)
>    passed  30/39  todo-test-todo-mark-unmark-category (0.087925 sec)
>    passed  31/39  todo-test-todo-quit01 (0.576536 sec)
>    passed  32/39  todo-test-todo-quit02 (0.160571 sec)
>    passed  33/39  todo-test-toggle-item-header01 (0.027441 sec)
>    passed  34/39  todo-test-toggle-item-header02 (0.124355 sec)
>    passed  35/39  todo-test-toggle-item-header03 (0.153855 sec)
>    passed  36/39  todo-test-toggle-item-header04 (0.203886 sec)
>    passed  37/39  todo-test-toggle-item-header05 (0.199669 sec)
> Items unarchived.
>    passed  38/39  todo-test-toggle-item-header06 (0.391414 sec)
>    passed  39/39  todo-test-toggle-item-header07 (0.098373 sec)
>
> Ran 39 tests, 38 results as expected, 1 unexpected (2022-10-12 09:37:34-0700, 
> 6.913405 sec)
>
> 1 unexpected results:
>    FAILED  todo-test-add-and-delete-file
>
>
>
>
>
>
>
> In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
>  3.24.34, cairo version 1.16.0) of 2022-10-05 built on naz
> Repository revision: e2027d60ba6c9a78ef9539234969ef2a44bf84fe
> Repository branch: my-build
> System Description: Debian GNU/Linux bookworm/sid
>
> Configured using:
>  'configure --with-native-compilation --with-pgtk'
>
> Configured features:
> ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
> JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP
> NOTIFY INOTIFY PDUMPER PGTK PNG RSVG SECCOMP SOUND SQLITE3 THREADS TIFF
> TOOLKIT_SCROLL_BARS XIM GTK3 ZLIB
>
> Important settings:
>   value of $LANG: en_US.UTF-8
>   value of $XMODIFIERS: @im=ibus
>   locale-coding-system: utf-8-unix
>
> Major mode: Fundamental
>
> Minor modes in effect:
>   pyvenv-mode: t
>   global-git-commit-mode: t
>   magit-auto-revert-mode: t
>   msb-mode: t
>   display-time-mode: t
>   global-tab-line-mode: t
>   tab-line-mode: t
>   shell-dirtrack-mode: t
>   auto-insert-mode: t
>   keyfreq-autosave-mode: t
>   keyfreq-mode: t
>   savehist-mode: t
>   icomplete-vertical-mode: t
>   icomplete-mode: t
>   editorconfig-mode: t
>   which-key-mode: t
>   electric-pair-mode: t
>   override-global-mode: t
>   tooltip-mode: t
>   global-eldoc-mode: t
>   show-paren-mode: t
>   electric-indent-mode: t
>   mouse-wheel-mode: t
>   tab-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   column-number-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   temp-buffer-resize-mode: t
>
> Load-path shadows:
> ~/env/elisp/ol-notmuch hides 
> /home/matt/.config/emacs/elpa/ol-notmuch-20220428.1337/ol-notmuch
> /home/matt/.config/emacs/elpa/transient-20220918.2101/transient hides 
> /home/matt/git/emacs-build/lisp/transient
>
> Features:
> (shadow emacsbug completion vc-annotate flyspell ispell perl-mode
> flymake-ruby flymake-easy ruby-mode whitespace conf-mode
> flymake-yamllint yaml-mode calc-alg calc-ext calc-menu calc
> calc-loaddefs calc-macs emoji-labels emoji multisession sqlite js qp
> pylint tramp tramp-loaddefs trampver tramp-integration tramp-compat
> ls-lisp yasnippet highlight-indentation elpy elpy-rpc pyvenv eshell
> esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module esh-groups
> esh-util elpy-shell elpy-profile elpy-django s elpy-refactor ido
> hideshow grep cus-edit cus-start cus-load sh-script executable textsec
> uni-scripts idna-mapping ucs-normalize uni-confusable textsec-check
> macros apropos emacs-news-mode rect cl-print shortdoc help-fns
> radix-tree magit-patch magit-subtree magit-gitignore magit-ediff ediff
> ediff-merg ediff-mult ediff-wind ediff-diff ediff-help ediff-init
> ediff-util term/xterm xterm git-rebase view facemenu two-column sort
> company-oddmuse company-keywords company-etags company-gtags
> company-dabbrev-code company-dabbrev company-files company-clang
> company-capf company-cmake company-semantic company-template
> company-bbdb company mail-extr files-x mm-archive tabify man notmuch
> notmuch-tree notmuch-jump notmuch-hello notmuch-show notmuch-print
> notmuch-crypto notmuch-mua notmuch-message notmuch-draft
> notmuch-maildir-fcc notmuch-address notmuch-company notmuch-parser
> notmuch-wash coolj goto-addr icalendar diary-lib diary-loaddefs
> notmuch-tag notmuch-lib notmuch-compat hl-line magit-extras face-remap
> magit-submodule magit-obsolete magit-blame magit-stash magit-reflog
> magit-bisect magit-push magit-pull magit-fetch magit-clone magit-remote
> magit-commit magit-sequence magit-notes magit-worktree magit-tag
> magit-merge magit-branch magit-reset magit-files magit-refs magit-status
> magit magit-repos magit-apply magit-wip magit-log which-func edebug
> magit-diff smerge-mode diff git-commit log-edit add-log magit-core
> magit-autorevert autorevert magit-margin magit-transient magit-process
> with-editor magit-mode transient magit-git magit-base crm dabbrev gud
> dired-aux misearch multi-isearch pulse display-line-numbers
> bug-reference protbuf msb time copyright mule-util markdown-mode color
> vc-hg vc-git diff-mode vc-bzr vc-src vc-sccs vc-svn vc-cvs vc-rcs
> log-view pcvs-util vc vc-dispatcher go-mode find-file etags fileloop
> eglot array jsonrpc ert pp ewoc debug backtrace xref flymake-proc
> flymake compile pcase imenu clang-format+ clang-format cap-words
> superword subword my llvm-c-style google-c-style cc-mode cc-fonts
> cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
> editorconfig-core editorconfig-core-handle editorconfig-fnmatch
> org-element avl-tree generator ol-w3m ol-rmail ol-mhe ol-irc ol-info
> org-habit org-agenda org-refile ol-gnus nnselect gnus-art mm-uu mml2015
> mm-view mml-smime smime gnutls dig gnus-sum gnus-group gnus-undo
> gnus-start gnus-dbus dbus gnus-cloud nnimap nnmail mail-source utf7 nnoo
> parse-time gnus-spec gnus-int gnus-range message sendmail yank-media
> rfc822 mml mml-sec epa derived epg rfc6068 epg-config mm-decode
> mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
> mailabbrev gmm-utils mailheader gnus-win ol-eww eww xdg url-queue shr
> pixel-fill kinsoku url-file svg xml dom puny mm-url gnus nnheader
> gnus-util text-property-search mail-utils range wid-edit mm-util
> mail-prsvr ol-doi org-link-doi ol-docview doc-view filenotify jka-compr
> image-mode exif dired dired-loaddefs ol-bibtex ol-bbdb tab-line server
> web-mode disp-table nix-mode ffap thingatpt smie nix-repl nix-shell
> nix-store magit-section dash compat-27 compat-26 nix-instantiate
> nix-shebang nix-format nix dirtrack ob-shell shell ob-ruby ob-python
> python compat compat-macs ob-dot org-protocol org ob ob-tangle ob-ref
> ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
> org-pcomplete pcomplete comint ansi-osc ansi-color ring org-list
> org-faces org-entities noutline outline org-version ob-emacs-lisp
> ob-core ob-eval org-table oc-basic bibtex iso8601 time-date org-keys oc
> org-loaddefs find-func cal-menu calendar cal-loaddefs finder-inf
> ol-notmuch ol org-compat org-macs format-spec skeleton autoinsert advice
> keyfreq project edmacro kmacro savehist icomplete comp comp-cstr
> warnings icons rx editorconfig which-key package browse-url url
> url-proxy url-privacy url-expand url-methods url-history url-cookie
> generate-lisp-file url-domsuf url-util mailcap url-handlers url-parse
> auth-source eieio eieio-core password-cache json subr-x map byte-opt
> url-vars cl-extra help-mode cl-macs gv cl-seq elec-pair use-package
> use-package-ensure use-package-delight use-package-diminish
> use-package-bind-key bind-key easy-mmode use-package-core cl-loaddefs
> cl-lib bytecomp byte-compile cconv info bazel-autoloads
> clang-format+-autoloads clang-format-autoloads cmake-mode-autoloads
> d-mode-autoloads debbugs-autoloads editorconfig-autoloads
> eglot-autoloads elpy-autoloads company-autoloads
> exec-path-from-shell-autoloads flymake-ruby-autoloads
> flymake-easy-autoloads flymake-yamllint-autoloads go-mode-autoloads
> google-c-style-autoloads graphviz-dot-mode-autoloads
> highlight-indentation-autoloads magit-autoloads git-commit-autoloads
> markdown-mode-autoloads meson-mode-autoloads nix-mode-autoloads
> magit-section-autoloads dash-autoloads nixpkgs-fmt-autoloads
> ol-notmuch-autoloads notmuch-autoloads orderless-autoloads
> org-drill-autoloads ox-hugo-autoloads persist-autoloads pylint-autoloads
> pyvenv-autoloads s-autoloads shfmt-autoloads reformatter-autoloads
> tomelr-autoloads transient-autoloads use-package-autoloads
> bind-key-autoloads vertico-autoloads web-mode-autoloads
> which-key-autoloads with-editor-autoloads compat-autoloads
> yaml-mode-autoloads yasnippet-autoloads rmc iso-transl tooltip eldoc
> paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
> mwheel term/pgtk-win pgtk-win term/common-win pgtk-dnd tool-bar dnd
> fontset image regexp-opt fringe tabulated-list replace newcomment
> text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
> isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
> font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
> indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
> tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
> romanian slovak czech european ethiopic indian cyrillic chinese
> composite emoji-zwj charscript charprop case-table epa-hook
> jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
> faces cus-face macroexp files window text-properties overlay sha1 md5
> base64 format env code-pages mule custom widget keymap
> hashtable-print-readable backquote threads dbusbind inotify
> dynamic-setting system-font-setting font-render-setting cairo gtk pgtk
> lcms2 multi-tty make-network-process native-compile emacs)
>
> Memory information:
> ((conses 16 1552683 200072)
>  (symbols 48 60189 405)
>  (strings 32 316503 22389)
>  (string-bytes 1 9599931)
>  (vectors 16 169883)
>  (vector-slots 8 3447448 320750)
>  (floats 8 774 987)
>  (intervals 56 68555 6744)
>  (buffers 1000 107))