bug#61846: Shell pathnames incorrectly handled, mostly due to hash characters

[John Cummings]

I use:  Editor:      GNU Emacs 28.2
        Hyperbole:   9.0.0
        Sys Type:    x86_64-pc-linux-gnu
        OS Type:     gnu/linux
        Window Sys:  None
        News Reader: Gnus v5.13
        Install:     elpa-devel, 8.0.1pre0.20230220.191327


Background:

This is usage of the Hyperbole action key on implicit buttons in a
shell buffer directory listing (bash/ls). Some file or directory names
appear to have unexpected results.


Expected Behavior (General):

Each file or directory in the listing will be interpreted as a
pathname implicit button (regardless of whether the name could also
match some other implicit button type in another context), and using
the action key on it will visit that file or directory. This will also
be reflected in the hkey-help output.


Actual Behavior (General):

Certain characters or patterns in names, and/or positioning of the
names in the listing, cause the names to be interpreted
incorrectly. Some names are interpreted incorrectly due to patterns in
their own name, and some due to patterns in other names in the
listing. That is, Hyperbole seems to be interpreting several names in
the listing as one composite name.

This can cause the Hyperbole action key to perform the wrong action,
like visitng the wrong file or directory, or not acting at all.

In some cases, the Hyperbole action key DOES visit the expected
location, but the hkey-help output still appears wrong.

The unexpected behaviors I saw were mostly related to the hash
character in names, which is not uncommon since Emacs creates those
files automatically for autosaving. But the behaviors are also
influenced by name positions relative to each other, meaning that
differently-sized windows could produce different behaviors. In this
test, I limited my window with to 80 characters, but the results would
be slightly different if the window was wider.

The main misinterpretations are:

- interpreting names with a hash character as a social media reference

- Names mistakenly interpreted as a subordinate of some other
  name in the listing, like it's a file in a subdirectory, or a link
  anchor in a pathname. This also appears to be linked to the
  appearance of the hash character in the name of the false parent.

- names not having any implicit button context, and no results when
  pressing the Hyperbole action key




Test Data:

I'll attempt to attach the test data directory as a tar archive, and
screenshots for more context. I'll also include the hkey-debug log
messages for all the action key presses.

The names I tested were shaped like this in my shell buffer:
'#buttons-9.txt#'   foofoofoo                      sources
 buttons-9.txt      hyperbole.el                   test.el
 buttons-9.txt~    '#*message*-20230220-192738#'   test.html
 buttons.txt       '#*message*-20230220-205601#'   test.html~
 buttons.txt~      '#*message*-20230220-210029#'   test.kotl
 EXAMPLE.kotl       source.el                      test.kotl~
 foo                source.el~
'#foofoofoo#'       source.elc



Specific Results:

(starting at the top left, going across to the right, and then
repeating for each row) Unless otherwise noted, the hkey-help output
was consistent with the action key behavior, whether the action key
behavior was right or wrong.

'#buttons-9.txt#' - Appears to be a non-functional social media
reference.

foofoofoo - Appears to be a child/anchor of '#buttons-9.txt' which
does not exist. So action key visits the data directory instead.

sources - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

buttons-9.txt - Same as foofoofoo above. However, the file being on the next
line appears to make the (incorrect) relationship of this file to its
false parent more interesting, inferring an underscore character.

hyperbole.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

test.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

buttons-9.txt~ - The action key opens this file as expected. However,
the hkey-help shows the incorrect lbl-key, just like buttons-9.txt. It
appears to interpret this name as a child/anchor of '#buttons-9.txt',
as well as adding an underscore

'#*message*-20230220-192738#' - Action key results in an error:
'(hpath:find): "#*message*-20230220-192738#" does not exist'. It
appears that the current directory is not applied to the find action.

test.html - Both hkey-help and pressing the action key say that there
is no matching context.

buttons.txt - Same as buttons-9.txt~ - Action key works, but hkey-help
is wrong, showing file as child/anchor of another name.

'#*message*-20230220-205601#' - same as previous mail message file -
does not include the current directory in the find action, so the find
raises an error.

test.html~ - Same as test.html above - no hkey-help or action button
context found.

buttons.txt~ - Same as buttons-9.txt~ - Action key works, but hkey-help
is wrong, showing file as child/anchor of another name.

'#*message*-20230220-210029#' - same as previous mail message file -
does not include the current directory in the find action, so the find
raises an error.

test.kotl - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

EXAMPLE.kotl - Same as buttons-9.txt above - child/anchor of
'#buttons-9.txt', with added underscore

source.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

test.kotl~ - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

foo - Same as buttons-9.txt above - child/anchor of '#buttons-9.txt',
with added underscore

source.el~ - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

'#foofoofoo#' - Same as '#buttons-9.txt#' above - nonfunctional social
media reference.

source.elc - Same as foofoofoo above - child/anchor of '#buttons-9.txt'



HyDebug messages from test session:
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key "#buttons-9.txt" lbl-start 36 
lbl-end 50 categ ibtypes::social-reference loc #<buffer *shell*> dir 
"/home/emacs/shellbug/data/" actype social-reference args ("" "#" 
"buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/foofoofoo" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/sources" categ ibtypes::pathname loc 
#<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file 
args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_buttons-9.txt" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/hyperbole.el" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/test.el" categ ibtypes::pathname loc 
#<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file 
args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_buttons-9.txt~" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/buttons-9.txt~    "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key "#*message*-20230220-192738#" 
lbl-start 173 lbl-end 200 categ ibtypes::pathname loc #<buffer *shell*> dir 
"/home/emacs/shellbug/data/" actype actypes::link-to-file args 
("#*message*-20230220-192738#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_buttons.txt" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/buttons.txt       "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key "#*message*-20230220-205601#" 
lbl-start 234 lbl-end 261 categ ibtypes::pathname loc #<buffer *shell*> dir 
"/home/emacs/shellbug/data/" actype actypes::link-to-file args 
("#*message*-20230220-205601#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_buttons.txt~" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/buttons.txt~      "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key "#*message*-20230220-210029#" 
lbl-start 296 lbl-end 323 categ ibtypes::pathname loc #<buffer *shell*> dir 
"/home/emacs/shellbug/data/" actype actypes::link-to-file args 
("#*message*-20230220-210029#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/test.kotl" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_EXAMPLE.kotl" categ 
ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/source.el" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/test.kotl~" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/_foo" categ ibtypes::pathname loc 
#<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file 
args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/source.el~" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key "#foofoofoo" lbl-start 431 
lbl-end 441 categ ibtypes::social-reference loc #<buffer *shell*> dir 
"/home/emacs/shellbug/data/" actype social-reference args ("" "#" "foofoofoo"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole 
button.  Available to action routines." lbl-key 
"/home/emacs/shellbug/data/#buttons-9.txt#/source.elc" categ ibtypes::pathname 
loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype 
actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))

test-data.tar
Description: Unix tar archive

screenshots.tar.xz
Description: application/xz