Re: Quoting special characters (was: Re: Possible solution for special characters in makefile paths)

[Frank Heckenbach]

Eli Zaretskii wrote:

> > So, while quoting may look natural in the makefile, it doesn't help with
> > the issues Frank raised regarding writing recipes correctly when paths
> > contain special characters: we'd be stripping that quoting before
> > running the recipe in any event.
> 
> First, we don't _have_ to strip them; we could keep the quotes.  And
> second, we could strip them, but record in the variable that it was
> quoted.  In both cases, we will restore the quoting in the context
> that needs them, such as when they are part of a shell command line.

The problem is how to quote them for a command line. In bash (and I
think any POSIX shell, but I'm not even sure of that), you can quote
an arbitrary string, but not in a uniform way. There are several
ways to do that, the simplest one (AFAIK) is to put '' around the
string and handle ' within the string specially (close the ', quote
the actual ' with a \, start a new ' quote for the rest of the
string, because \ does not escape within ''; therefore the
Q='$(subst ','\'',$(1))' in my previous mail).

In other shells things are probably different (if possible at all).
I think it's different in csh, and you probably know better about
Windows shells. So what make would have to do would depend on
$(SHELL) which Paul said he doesn't like, and it could only ever be
a heuristic anyway. (What if SHELL is set to foobar which might be a
wrapper around bash, csh or some other shell?)

Yet, as I said, I'd like to see a solution to this problem. So even
if it's not possible in a fully automatic way, the next best thing
is to require as little user intervention as possible. This might be
achieved with a special variable, let's call it SHELL_QUOTE, which
has a similar status as SHELL and is applied when expanding a
variable into a command line -- separately for each word of it(!),
and of course also for automatic variables such as $@, $< or $^.
E.g. when you have two files called "foo bar" and "baz qu'x"
(including spaces and single quote, not including the double
quotes), $(wildcard *) would yield a string consisting of two words,
internally escaped e.g. as "foo\001bar baz\001qu'x", and when put in
a command line, make would undo the escaping and apply SHELL_QUOTE
to each word of it resulting in 'foo bar' 'baz qu'\''x' which can
then be passed to bash.

Unfortunately, SHELL_QUOTE couldn't be set by default because it
would break commands with manual quoting like:

foo:
        touch "$@"

(Unless make would apply SHELL_QUOTE only if necessary, but that
raises the question when it's necessary which depends of which
characters are special to the given shell and starts getting
heuristic again ...)

But at the worst, users would need one setting of SHELL_QUOTE for
each kind of shell, independent of the individual makefile, so this
knowledge could spread around for common shells.

> > Remember we're not just talking about paths appearing in target
> > and prerequisite lists where we at least have some context and can
> > perhaps expect that the special characters are uncommon: we must also
> > consider variable values which have no context: some may be used as
> > target/prerequisites, but others will just be used in a recipe for
> > completely unrelated purposes.  And, due to make functions, we want to
> > chop these variable values up into words _all the time_.  That
> > word-chopping internal to make must take into account quoting/escaping.
> 
> If we know that a variable was quoted, we will treat it as a single
> word.  So I don't really see any serious problems here (of course,
> this special handling of quoted strings will have to be coded).

Unfortunately, it's not so easy. In my example above, the value
could neither be words-separated-by-spaces like now (would become 4
words), nor a single word as you suggest. What they really are is a
list of strings (that each may contain spaces), but storing and
handling them this way is probably the major change in the string
manipulation that Paul is afraid of. His suggestion of some internal
escaping mechanism would avoid such changes, because internally then
a space would be encoded by some other character, and word-splitting
by spaces (like now) would still work.

> > Today if you have:
> >     FOO := "foo "bar "biz
> >     BAR := $(addsuffix ",$(FOO))
> > 
> > the result will be: "foo" "bar" "biz".  How do we deal with this if
> > quoting is special to make?
> 
> Why should we bother about these hypothetical use cases?  Are there
> any real-life Makefiles that do anything like that?  If there are,
> let's look at them, and let's understand why they need this quote
> juggling in the first place.

I guess that's why he asked us for such examples. I don't have one,
but considering the uproar we saw last about a trivial change WRT
"mixed implicit and normal rules", I can understand why he's careful
about the slightest backward-incompatibility.

> > Using backslashes has similar problems, especially on Windows
> > systems where you might expect backslashes to be used in variables
> > as a matter of course.
>
> No, there are no problems on Windows, because file or directory names
> on Windows do not _start_ with blanks.

Is this actually forbidden or just by convention?

Even if so, what about a case like this:

foo:
        rsync c:\foo\ c:\bar\

(Of course, it's a made-up example. rsync was the first thing that
came to my mind where a trailing directory separator makes an
important difference, and I wrote what I imagine it might look like
with Windows paths.)