Re: Autoconf Digest, Vol 129, Issue 1

[David A. Wheeler]

Bob Friesenhahn:
> It is a common problem for Windows users that their default directory 
(equivalent to a home directory) has spaces in it.

In recent versions of Windows the home directory is often c:\Users (and thus 
has no space). But the directory that contains most programs has a space in its 
name:  "C:\Program Files".





On January 1, 2015 12:55:57 AM EST, address@hidden wrote:
>Send Autoconf mailing list submissions to
>       address@hidden
>
>To subscribe or unsubscribe via the World Wide Web, visit
>       https://lists.gnu.org/mailman/listinfo/autoconf
>or, via email, send a message with subject or body 'help' to
>       address@hidden
>
>You can reach the person managing the list at
>       address@hidden
>
>When replying, please edit your Subject line so it is more specific
>than "Re: Contents of Autoconf digest..."
>
>
>Today's Topics:
>
>   1. Re: Mangled argument vector choking on spaces? (David A. Wheeler)
>   2. Re: Autoconf Digest, Vol 128, Issue 6 (David A. Wheeler)
>   3. Re: Mangled argument vector choking on spaces? (Fotis Georgatos)
>   4. Re: Mangled argument vector choking on spaces? (Paul Eggert)
>   5. Re: Mangled argument vector choking on spaces? (David A. Wheeler)
>   6. Re: Mangled argument vector choking on spaces? (Bob Friesenhahn)
>   7. Re: Mangled argument vector choking on spaces? (Paul Eggert)
>   8. Re: Mangled argument vector choking on spaces? (David A. Wheeler)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Wed, 31 Dec 2014 12:02:19 -0500 (EST)
>From: "David A. Wheeler" <address@hidden>
>To: "eggert" <address@hidden>
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset="utf-8"
>
>I wrote:
>> > SBCL =/cygdrive/c/"Program Files/Steel Bank Common Lisp/1.2.6/sbcl"
>
>On Tue, 30 Dec 2014 23:38:20 -0800, Paul Eggert <address@hidden>
>replied:
>> That wouldn't work for commands that do stuff like this:
>>     $(MAKE) SBCL="$(SBCL)"
>
>Sure, but this minor variation WOULD work in a makefile, and it's an
>easy fix:
> $(MAKE) SBCL='$(SBCL)'
>
>You can also embed double-quotes when setting environment variables and
>invoking make directly:
>ENV_SBCL='/cygdrive/c/"Program Files/Steel Bank Common
>Lisp/1.2.6/sbcl"' make demo3
>
>Another possible option (noted below) is to use single quotes instead
>of double quotes.
>Maybe that would be the "easiest way forward".
>
>
>> There is no simple and clean solution to this problem, I'm afraid.
>
>That may be true, but maybe we can find "simple and clean enough".
>There are really only 3 main options that I see to disambiguate
>"spaces used as argument separators" and "spaces in pathnames" in
>input:
>1. backslash prefixing (but this is heavily overloaded because this
>data goes a varying number of unquoting processes),
>2. double quotes
>3. single quotes
>
>
>> The best place to start fixing it would be with GNU Make...
>
>But that only works with GNU make.  GNU make is great, but it's not the
>only make.
>Autoconf (and automake) are supposed to work with other makes, so any
>solution needs to *NOT* depend on GNU make.
>
>> In the meantime, people will just have to live with not having
>> white space in their development sources' directory names.
>
>The problem is that that's not the problem :-).
>I agree that developers should avoid having whitespace in their source
>dirnames;
>typically this is easy to do.
>
>The problem is that in many *USER* (builder) environments,
>tools and configuration information have pathnames with spaces.
>This is NOT the developer's environment at all, and thus the
>developer CANNOT control this.  Users can't control it either,
>actually; it is
>a current fact of their environment, especially if they use Mac OS X or
>Windows,
>and not something they can change.
>They can do workarounds with symlinks, etc., to try to *hide* the
>reality, but
>since autoconf is supposed to do things *automatically*, that means
>that
>autoconf fails to do the one job it's supposed to do: automatic
>configuration.
>
>--- David A. Wheeler
>
>
>
>------------------------------
>
>Message: 2
>Date: Wed, 31 Dec 2014 12:19:37 -0500 (EST)
>From: "David A. Wheeler" <address@hidden>
>To: "autoconf" <address@hidden>
>Subject: Re: Autoconf Digest, Vol 128, Issue 6
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset="utf-8"
>
>Bob Friesenhahn <address@hidden>
>> The POSIX-style shell tosses the double quotes at first point of use.
>
>When a variable reference is itself "..." quoted, like "$variable",
>a POSIX shell does NOT toss the double-quotes.
>When it's unquoted, a POSIX shell *does* toss the double-quotes.
>I am *counting* on that behavior to make this scheme work.
>
>This works out in makefiles at least.
>Going back to my examples in a makefile:
>SBCL =/cygdrive/c/"Program Files/Steel Bank Common Lisp/1.2.6/sbcl"
>EGREP = /usr/bin/grep -E
>
>Typically programs are invoked in a makefile as $(NAME),
>and this 'just works' in the proposed convention:
>dostuff:
>  $(SBCL) < x.src > x.dest
>  $(EGREP) pattern y.src >> x.dest
>
>
>> Arguments must be escaped varying amounts of times to survive a given
>
>> data path, and the data path may not be a fixed one.
>
>Many shell programmers, out of habit,
>always reference shell variables as "$variable_name" instead of
>$variable_name,
>and then quoting survives just fine.  It might not be hard to fix up
>the
>remaining ones (it is a good habit anyway).
>
>There might need to be a way to vary how replacements are done
>(e.g., a Makefile would have the quoting scheme, while a #include file
>would not).
>But the fundamental problem is that there's no way to distinguish the
>different uses of spaces; once there's an input and storage convention,
>you can then decide how to best output them.
>
>--- David A. Wheeler
>
>
>
>------------------------------
>
>Message: 3
>Date: Wed, 31 Dec 2014 19:36:41 +0200
>From: Fotis Georgatos <address@hidden>
>To: <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset="windows-1252"
>
>
>Hello * and a happy new year,
>
>As regards $subject, this so-called STRING_ENCODING_CHARMAP may be
>relevant:
>https://github.com/hpcugent/easybuild-framework/blob/master/easybuild/tools/filetools.py#L59
>Look around line L1000 for the rational of that design and some
>pointers.
>
>Basically, ` ` becomes `_space_` and so on, for many more potentially
>tricky characters.
>The aim was to have freedom in relation to packages/filenames. It seems
>to work OK! (*)
>
>On Dec 31, 2014, at 7:00 PM, <address@hidden>
><address@hidden> wrote:
>> I propose developing and documenting a standard convention for spaces
>in pathnames that
>> focuses on making things easier for *users* - the builders of the
>software.
>> This convention could be enabled by some statement in autoconf.ac,
>and perhaps
>> it could be enabled by default some time in the future.
>> If autoconf defined a simple convention for differentiating spaces in
>pathnames
>> from spaces in argument separators, it might "just work".  It'd be
>tricky to find such a convention,
>> but if one can be found, autoconf would be able to adapt to many
>common systems.
>
>(*)
>
>To make a long story short, we faced the same problem during EasyBuild
>open source project, for building software intended for HPC platforms; 
>in fact, that one issue triggered me to request EB?s first hackathon
>whereby,
>in the issued Pull Request, I contained the wishes of a handful of
>people.
>It is not known if there are better ways to go about it - and if so,
>what?s that?
>
>We opted for that solution as the least hackish or, least worst.
>It does not preserve 1-1 character mapping but IMHO no generic solution
>would do that, anyhow. I hope you find this technique useful/relevant.
>
>enjoy,
>Fotis
>
>
>ps.
>If you know *anybody* who has setup a database of build
>modes/instructions
>for 1000s of software packages across architectures & OS platforms,
>kindly pass on the information! (ie. something more reusable than
>pkgsrc)
>
>N.B.
>autoconf build processes are handled out-of-the-box by EB, here is an
>example:
>https://github.com/hpcugent/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/a/ABySS/ABySS-1.3.6-goolf-1.4.10-Python-2.7.5.eb
>
>If you wonder why we bother with yet another build system, obviously,
>you haven?t been debugging HPC environments for 100s of users; have fun
>reading this:
>https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/b/boost.py
>
>
>-- 
>echo "sysadmin know better bash than english" | sed s/min/mins/ \
>  | sed 's/better bash/bash better/' # signal detected in a CERN forum
>
>
>
>
>
>
>
>
>
>
>------------------------------
>
>Message: 4
>Date: Wed, 31 Dec 2014 11:31:58 -0800
>From: Paul Eggert <address@hidden>
>To: address@hidden
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset=utf-8; format=flowed
>
>David A. Wheeler wrote:
>
>> Sure, but this minor variation WOULD work in a makefile, and it's an
>easy fix:
>>   $(MAKE) SBCL='$(SBCL)'
>
>That won't work if SBCL contains single quotes, another common
>practice.
>
>>> The best place to start fixing it would be with GNU Make...
>>
>> But that only works with GNU make.
>
>GNU make would be just the first step.  The next step would be to get
>the fix 
>into POSIX and into other 'make' implementations.  This sort of thing
>has been 
>done before.
>
>> They can do workarounds with symlinks
>
>How about fixing Autoconf to create a symlink from /tmp to srcdir if
>srcdir 
>contains a space, and using the symlink instead?  That should work too.
> Please 
>feel free to propose a patch along those lines.
>
>
>
>------------------------------
>
>Message: 5
>Date: Wed, 31 Dec 2014 15:39:43 -0500 (EST)
>From: "David A. Wheeler" <address@hidden>
>To: "eggert" <address@hidden>
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset="utf-8"
>
>David A. Wheeler wrote:
>> > Sure, but this minor variation WOULD work in a makefile, and it's
>an easy fix:
>> >   $(MAKE) SBCL='$(SBCL)'
>
>On Wed, 31 Dec 2014 11:31:58 -0800, Paul Eggert <address@hidden>
>wrote:
>> That won't work if SBCL contains single quotes, another common
>practice.
>
>That's pretty rare in my experience.  In any case,
>that is just the symptom of the problem.  Because there is no standard
>way to distinguish spaces as argument separators vs. spaces in
>pathnames,
>people feel free to do anything.  By defining a convention, people can
>use the
>sequence in the ways intended, and avoid them for other reasons.
>
>> GNU make would be just the first step.  The next step would be to get
>the fix 
>> into POSIX and into other 'make' implementations.  This sort of thing
>has been 
>> done before.
>
>That certainly sounds promising.  What's the plan?  I know there was
>some
>discussion about how to handle spaces in GNU make, but I haven't been
>following it closely
>for a while.
>
>> How about fixing Autoconf to create a symlink from /tmp to srcdir if
>srcdir 
>> contains a space, and using the symlink instead?  That should work
>too.  Please 
>> feel free to propose a patch along those lines.
>
>That is irrelevant for my use case, if I understand you correctly.
>In all the cases I'm concerned about, spaces are *NOT* in any path
>inside a source directory srcdir.  So "fixing the distributed
>filenames"
>or "creating a symlink to srcdir" does NOTHING useful for my users.
>
>In all cases I'm concerned about, the spaces are the correct results
>of system probing by "./configure"; these values are NOT in any sense
>within srcdir.
>E.G., when installing an application that requires sbcl, the
>autoconf-generated
>"configure.ac" needs to find where sbcl is installed.  On Windows
>systems
>with Cygwin that value will normally be this (note the spaces):
>  /cygdrive/c/Program Files/Steel Bank Common Lisp/1.2.6/sbcl
>Thus, autoconf+automake might put this in the Makefile:
>  SBCL = /cygdrive/c/Program Files/Steel Bank Common Lisp/1.2.6/sbcl
>Similarly, autoconf+automake might decide to put this in the Makefile
>for EGREP:
>  EGREP = /usr/bin/grep -E
>The cause is that the space character has two *incompatible* meanings
>when autoconf
>returns probe results.
>These makefile values need to be used during build, and often end up
>getting
>stored in generated executables that are then shared across the system.
>Creating a symlink from /tmp to srcdir does nothing useful in these
>cases.
>
>--- David A. Wheeler
>
>
>
>------------------------------
>
>Message: 6
>Date: Wed, 31 Dec 2014 16:28:46 -0600 (CST)
>From: Bob Friesenhahn <address@hidden>
>To: "David A. Wheeler" <address@hidden>
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID:
>       <address@hidden>
>Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
>On Wed, 31 Dec 2014, David A. Wheeler wrote:
>
>>> How about fixing Autoconf to create a symlink from /tmp to srcdir if
>srcdir
>>> contains a space, and using the symlink instead?  That should work
>too.  Please
>>> feel free to propose a patch along those lines.
>>
>> That is irrelevant for my use case, if I understand you correctly.
>> In all the cases I'm concerned about, spaces are *NOT* in any path
>> inside a source directory srcdir.  So "fixing the distributed
>filenames"
>> or "creating a symlink to srcdir" does NOTHING useful for my users.
>
>It is a common problem for Windows users that their default directory 
>(equivalent to a home directory) has spaces in it.
>
>Bob
>-- 
>Bob Friesenhahn
>address@hidden,
>http://www.simplesystems.org/users/bfriesen/
>GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/
>
>
>
>------------------------------
>
>Message: 7
>Date: Wed, 31 Dec 2014 18:03:09 -0800
>From: Paul Eggert <address@hidden>
>To: address@hidden
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset=utf-8; format=flowed
>
>David A. Wheeler wrote:
>> David A. Wheeler wrote:
>
>>> That won't work if SBCL contains single quotes, another common
>practice.
>>
>> That's pretty rare in my experience.
>
>Shrug.  I do it all the time.  I prefer single-quotes, anyway, as
>they're safer 
>for shell quoting.
>
>> I know there was some
>> discussion about how to handle spaces in GNU make, but I haven't been
>following it closely
>> for a while.
>
>Likewise.  But that's the plan, anyway.
>
>> E.G., when installing an application that requires sbcl, the
>autoconf-generated
>> "configure.ac" needs to find where sbcl is installed.  On Windows
>systems
>> with Cygwin that value will normally be this (note the spaces):
>>    /cygdrive/c/Program Files/Steel Bank Common Lisp/1.2.6/sbcl
>
>Sure.  So create a symlink to that, e.g.,
>
>ln -s '/cygdrive/c/Program Files/Steel Bank Common Lisp/1.2.6/sbc'
>/tmp/xyz
>
>Then configure with the equivalent of SBCL=/tmp/xyz.
>
>Obviously the idea needs some elaboration (e.g., if the file name is
>needed at 
>runtime) but it would work.  All it needs is Somebody to write and test
>and 
>document the patch.
>
>
>
>------------------------------
>
>Message: 8
>Date: Thu, 01 Jan 2015 00:55:36 -0500
>From: "David A. Wheeler" <address@hidden>
>To: Paul Eggert <address@hidden>
>Cc: autoconf <address@hidden>
>Subject: Re: Mangled argument vector choking on spaces?
>Message-ID: <address@hidden>
>Content-Type: text/plain; charset=UTF-8
>
>Paul Eggert:
>>Sure.  So create a symlink to that, e.g.,
>>   ln -s '/cygdrive/c/Program Files/Steel Bank Common Lisp/1.2.6/sbc'
>/tmp/xyz
>> Then configure with the equivalent of SBCL=/tmp/xyz.
>> Obviously the idea needs some elaboration (e.g., if the file name is
>needed at 
>runtime) but it would work.  All it needs is Somebody to write and test
>and 
>document the patch.
>
>That will not work well in many cases.  /tmp often doesn't stick
>around, nor is there any standard safe system-wide place anything could
>write to.  How do you handle containers where the software is
>preinstalled and provided on readonly mounts and with new blank /tmp? 
>This approach is also nasty to package.  The whole approach is a weird
>kludge.
>
>I do grant you points for being clever, but isn't there a better way?
>
>--- David A.Wheeler
>
>------------------------------
>
>_______________________________________________
>Autoconf mailing list
>address@hidden
>https://lists.gnu.org/mailman/listinfo/autoconf
>
>
>End of Autoconf Digest, Vol 129, Issue 1
>****************************************

--- David A.Wheeler