Re: Quoting special characters (was: Re: Possible solution for special characters in makefile paths)

[Eli Zaretskii]

> From: Paul Smith <address@hidden>
> Cc: address@hidden
> Date: Sat, 22 Feb 2014 10:23:33 -0500
> 
> So the first thing to decide is whether the quoting exists only internal
> to make and is removed before the string is passed to any external
> interface (shell, stat(), stdout, environment, etc.), or if the quoting
> will be preserved and passed along to the commands make invokes (not
> stat() obviously).
> 
> I think it's clear that the solution MUST only be internal to make.

I'm not sure I understand what this means, in practice.  A quoted
string will certainly come from a Makefile, and must be quoted there
already.  Whether it should keep the quoting in memory is a separate
issue, whose resolution is a practical matter, not a theoretical one
-- we should resolve it according to the merits and demerits of each
of the two alternatives.

> Regardless of the quoting behaviors make employs, we cannot be sure that
> whatever utility we use to run recipes will treat quotes the same way.
> Users can choose whatever SHELL value they want, for example, and I
> certainly would not want to provide a solution that worked only when a
> POSIX-y shell was used.

That's not our problem.  This is a problem for the person who writes
the Makefile which uses such strings.  We CANNOT solve this problem
inside Make, so we shouldn't even try.  You already mentioned the
problems with quoting in command lines -- these are unsolvable on the
Make level.  Only the user knows how to solve them.

> So, while quoting may look natural in the makefile, it doesn't help with
> the issues Frank raised regarding writing recipes correctly when paths
> contain special characters: we'd be stripping that quoting before
> running the recipe in any event.

First, we don't _have_ to strip them; we could keep the quotes.  And
second, we could strip them, but record in the variable that it was
quoted.  In both cases, we will restore the quoting in the context
that needs them, such as when they are part of a shell command line.

> Next lets consider the parsing of these values.  Here I think trying to
> re-use any existing quoting runs into very significant problems.  There
> may be a few handfuls of makefiles that use the mapping special
> characters, but there must be many thousands of makefiles that include
> quotes and/or backslashes today.

True; but the semantics of these quotes in the existing makefiles can
be nothing except what we want in this case, anyway.  After all, Make
is a utility for invoking commands with arguments, where quote
characters have a very specific meaning and purpose.

> Remember we're not just talking about paths appearing in target
> and prerequisite lists where we at least have some context and can
> perhaps expect that the special characters are uncommon: we must also
> consider variable values which have no context: some may be used as
> target/prerequisites, but others will just be used in a recipe for
> completely unrelated purposes.  And, due to make functions, we want to
> chop these variable values up into words _all the time_.  That
> word-chopping internal to make must take into account quoting/escaping.

If we know that a variable was quoted, we will treat it as a single
word.  So I don't really see any serious problems here (of course,
this special handling of quoted strings will have to be coded).

> Today if you have:
> 
>     FOO := "foo "bar "biz
>     BAR := $(addsuffix ",$(FOO))
> 
> the result will be: "foo" "bar" "biz".  How do we deal with this if
> quoting is special to make?

Why should we bother about these hypothetical use cases?  Are there
any real-life Makefiles that do anything like that?  If there are,
let's look at them, and let's understand why they need this quote
juggling in the first place.

> Using backslashes has similar problems, especially on Windows
> systems where you might expect backslashes to be used in variables
> as a matter of course.

No, there are no problems on Windows, because file or directory names
on Windows do not _start_ with blanks.

> Today make does not reserve backslashes at all UNLESS they're
> quoting a character that's special to make... and whitespace isn't.

Well, whitespace _will_ be special from now on.  Where's the problem
with that?

> Personally, I can't begin to articulate a coherent set of rules that
> would allow this to work... but I'm certainly open to the idea that I'm
> missing something.

You didn't convince me there is a problem.  Maybe a few "sane"
examples will, but I haven't seen them till now.

> If make had a coherent backslash escaping mechanism (for example,
> something like the shell where backslash always escapes the next
> character and if you want a backslash you must always write "\\") then
> we would be all set.  But instead make's implementation tries to be too
> smart and only interprets backslashes as escapes when they appear in the
> context of a special character...

Exactly my point: using a backslash to escape a blank is a natural
extension of what we already do with a colon.  I fail to see why is it
so different.

> special character without enormous backward-compat problems!  Consider
> how many variable values out there are using backslashes to quote
> whitespace in preparation for the shell, that would have to be changed!
> Consider:
> 
>     SWITCH = switch "$(FOO)" in (foo\ bar) echo hi ;; esac
> 
> !!

How is this a problem?  Or are you saying that the backslash here does
something other that quote the following blank?  If so, what _does_ it
do?