Re: Quoting special characters (was: Re: Possible solution for special characters in makefile paths)

[Frank Heckenbach]

Paul Smith wrote:

> On Tue, 2014-02-25 at 17:09 +0100, Frank Heckenbach wrote:
> > > > In the command-line it would also substituted as is, so no change from
> > > > the existing behaviour here.
> > > 
> > > Ouch; this statement is not precise and this is THE CRUX of the problem.
> > > What exactly do you mean by "substituted as is"?  As is what?
> > 
> > As it is stored internally, i.e. with the backslash.
> 
> Just to be clear I don't want to say that this is how things will be
> stored internally.  Maybe it will be, maybe not.  Eli has a great point
> that we don't need to bind these things together too tightly during the
> design discussion and the most important part is the user interface.
> However, I understand that what you mean is "as it was written in the
> makefile".

Of course, make can store strings internally as it pleases as long
as it doesn't "leak". I think I see why you prefer other encodings
(not having to change the word splitting routines). I don't know the
code as well as you do, so I can't tell how much effort it really
would be. I just think not reencoding might result in cleaner code
in the end.

Especially if we want to support different forms of quoting ("", ''
and \) and preserve them (see below why I think we should),
reencoding becomes more and more difficult.

> > Yes. Actually, I think/fear make will not only have to deal with
> > backslashes in word-splitting contexts, but with the full glory of
> > quoting as supported by POSIX sh, including single and double
> > quotes, so $^ will actually mean the same to the shell than it did
> > to make in the dependency list.
> 
> Why do you think this will be needed?

When the makefile explicitly single-/double quotes stuff:

FOO = a'()'b
all: $(FOO); cat $(FOO)

If make doesn't handle ', it will depend on a file called «a'()'b»
including quotes, and give this on the command line, which to the
shell means «a()b», i.e. a different file name.

Of course, there are different possible solutions to this problem:

What I suggest is that make just deals with ' like the shell, i.e.
stores the string as «a'()'b», gives this on the command line, and
unquotes it according to shell rules (resulting in «a()b») in the
dependency list.

Alternatively, it could change quotes to backslashes on sight, i.e.
store the string as «a\(\)b». This seems to be what you prefer (or
rather, another encoding internally, and backslashes on output). I
think it would work, but it would "leak" (the echoed command would
look less similar to what's written), and I don't actually think it
would be much less work: You still need to do with shell-style
quoting, even if only in one place (parsing).

A related question is what to do with unmatched quotes:

FOO = '

We cannot forbid them entirely, since this is valid now and should
remain valid:

all: ; echo $(FOO)Hello world$(FOO)

But I think that's ok. As long as we store the string as is (my
proposal), we don't have a problem here.

We can/must forbid such a string in target/dependency lists because
it doesn't make sense:

all: $(FOO)
*** error: invalid quoted string in dependency list

> > > > That's probably not what was intended, but I'd blame this on the
> > > > users.
> > > 
> > > Well, I think it's a little harsh to blame the users for doing what 40
> > > years of make history and portability have taught them is best-practice.
> > 
> > Is this so? I've seen a lot of makefiles that don't quote address@hidden 
> > (Then
> > again, I've seen a lot of shell scripts that don't quote their
> > variables when they really should.) So probably the best we can say
> > it that there's a lot of mess out there. There's no hope to make all
> > of it work correctly, so we can only choose one variant we want to
> > make work.
> 
> I'm not saying that we won't have to change something.  I'm just saying
> it's not fair to blame the user for not writing their makefile
> "correctly".  We have to own up to the fact that we're introducing a
> backward-incompatibility and hope people feel that the increased utility
> is worth the pain of modifying their (previously perfectly legitimate)
> makefiles.

As long as they use them with harmless characters, the makefiles
remain valid because the additional quoting won't hurt. And that's
basically all you can do today without tricks(*), so the
incompatibility might not be so bad. For new makefiles intended to
support file names with spaces, users will just have to learn not to
quote manually.

(*) That's of course the tricky part. ;) As you asked, which kind of
    tricks do people do today which might break then. I don't know.