Re: Unclosed quotes on heredoc mode

[Robert Elz]

    Date:        Wed, 17 Nov 2021 18:45:05 +0000
    From:        =?utf-8?Q?Jo=C3=A3o_Almeida_Santos?= 
<j-almeidasantos@hotmail.com>
    Message-ID:  
<VI1PR01MB6896EAEBDA5A8112C23C4E0EF59A9@VI1PR01MB6896.eurprd01.prod.exchangelabs.com>

  | No, it's on the email...

It wasn't, but some lists filter attachments (remove them) - this might be one.

  | bash-5.1$ echo $PATH
  | 
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin/:/usr/local/bin/:/usr/local/bin/

That's fine, but having /usr/local/bin there 3 times isn't useful for
anything but wasting time.

  | bash-5.1$ cat << $PATH


  | it should have terminated with the upper delimiter!

What do you consider the "upper delimiter" ?

This is one of the weirder aspects of shell syntax, and perhaps one
of bash's oddities.

  | but, bash does not seem to expand PATH.

Which one are you concerned with?   It really should expand neither
reference to $PATH in that example - the word after << is just a string,
it isn't used anywhere except locating the end of the here doc data.
However, that word can sometimes be expanded (by some shells), whereas
all that should happen to it is quote removal.   Avoiding '$' in that
word is a very good idea.

The same is true of the '$PATH' that ends the here document, that's also
just a string - which is also never expanded (no-one should be expanding
that one, ever).

The only point of this string (where you're using $PATH) is as a marker
to show where the here document data ends, it does nothing else at all.
In some instances (where the data might contain just about anything) a
complicated end word (so it is unlikely to be a line in the here document
data) is needed, in other cases something simple like EOF or DONE or my
favourite: !  is all that is needed.

So
        cat <<!
        $PATH
        !

(all entered starting in column 1) will expand PATH, place its value
as a line on standard input for cat, which will read that, and print it.
The '!' characters do nothing else than indicate where the here document
ends, the one after the << just tells the shell what string to look for.


  | If I use unclosed quotes on heredoc, I can't use 
  | the given delimiter to end the heredoc, I end up having to use an EOF.

First, the EOF should not work, that's a bash bug (IMO) - that should
generate an error, not just a warning.   But that is beside the point.

 Example:
  |
  | bash-5.1$
  | bash-5.1$ cat << ola"

OK, here we have another of the oddities of shell syntax.   The spec
says that a here document starts at the next newline after the << operator,
but that's not what it really means.  The newline that occurs after the "
there does not count, that newline is just a character in the end delimiter
word.

So, to repeat that line again, with the following ones:

  | bash-5.1$ cat << ola"
  | > I,
  | > ola""

That first " in that last string closes the opening " in the end delimiter
word.   The second " in that line starts a new quoted string, which is
still part of the same word (no separators have occurred)

  | bash-5.1$ cat << ola"
  | > I,
  | > ola""
  | > ola"

Now we have a closing " on for the opening (2nd) one on the previous
line, and the newline that follows that ends that word, and also becomes
the newline after which the here document data starts.  So now the end
delimiter word is complete, it is, as a C string:

        "old\nI,\nola\nola"

The internal quotes have been removed, they do not form part of it,
the ones added here are just for clarity, and don't form part of the word.
The \n sequences represent actual newline characters, not the 2 character
sequence '\' 'n'.

That string is what you would need to use, on a line, to end the
here document.   Being able to do that (include embedded newline characters
in a "line") isn't required by the shell specification, and (it has been
a while since I checked) I do not believe that those work in bash (they
do in some other shells).

That means that (in standard shell required syntax, and probably in
bash) there is nothing you can possibly enter which will terminate that
here document correctly.   So an end of file, or interrupt (^C probably)
is your only choice.


  | In the above example, I don't unterstand how to provide the wanted
  | delimiter!

You cannot in bash, I believe, it is simply impossible.

  | I try to close the quote, in case it is needing it, but even with both
  | quotes, just one or none, it doesn't close...not even with a '\n' char

In shell, newlines do not end quoted strings, they just become a character
in the string.   The word that comes after the << cannot (in standard syntax)
contain a newline, so you cannot use an unpaired quote character in it
(unless it is itself quoted - and if you do that, nothing in the here doc
will be able to be expanded).

Since bash doesn't allow end delimiter words that contain newlines to
work, it should probably generate an error when you try to use one, that
would have made things clear.

If you wanted a quote character in the end delimiter word for some
reason, you could do:

        cat << ola\"
        hello
        old"

and that should say "hello" on standard output, as the end delimiter there
is the 4 char string:
        'o' 'l' 'a' '"'
but no expansions in the "hello" part can happen, because of the quoting
used in the end delimiter word.

kre