Re: [lmi] Strange LMI installation failure under Windows 7 SP1: /cache_for_lmi/downloads not found

[Greg Chicares]

On 2014-08-20 14:47Z, Vadim Zeitlin wrote:
> On Wed, 20 Aug 2014 14:01:52 +0000 Greg Chicares <address@hidden> wrote:
> 
> GC> On 2014-08-18 16:30Z, Vadim Zeitlin wrote:
[...]
> GC> > Also, further down in the log there is
> GC> > 
> GC> >         cd /cache_for_lmi/downloads && echo 
> "6bba3bd1bf510d152a42d0beeeefa14d *binutils-2.19.1-mingw32-bin.tar.gz" | 
> md5sum --check
> GC> >         /bin/sh: line 0: cd: /cache_for_lmi/downloads: No such file or 
> directory
> GC> >         install_mingw.make:251: recipe for target 
> 'binutils-2.19.1-mingw32-bin.tar.gz' failed
> GC> >         make: *** [binutils-2.19.1-mingw32-bin.tar.gz] Error 1
> GC> > 
> GC> > However the next command does find the directory successfully:
> GC> > 
> GC> >         cd /cache_for_lmi/downloads && echo 
> "ada423c49bc5bfa7c3e7a80a711c2a1a *mpatrol_1.4.8.tar.gz" | md5sum --check
> GC> >         mpatrol_1.4.8.tar.gz: OK
> GC> 
> GC> So 'cd' fails at first, but later succeeds with the same directory.
> 
>  This is indeed what happens and I now know the reason:
> install_mpatrol.make explicitly creates this directory in its initial_setup
> target, which is a dependency of "all". However install_mingw.make does not
> do this, and as the directory doesn't exist initially (see below for why),
> it fails.

Just to be sure...are you following the steps in 'INSTALL'? Its first steps are:

| Open a "Command Prompt" window, and enable pasting into it:
...
| Copy and paste the following lines into the "Command Prompt" window:
|
|   C:
|   mkdir C:\cache_for_lmi

By this point, 'C:\cache_for_lmi' must exist--unless msw 'mkdir' failed,
but you would have noticed that.

Then 'install_cygwin.bat' is run in that CMD session; one of its commands is:

echo C:/cache_for_lmi          /cache_for_lmi lmi_specific binary,user 0 0 >> 
fstab

Now "C:/cache_for_lmi" must exist, and writing this line to /etc/fstab will
cause that directory to be mounted when Cygwin is started. If you're following
only the steps in 'INSTALL', then Cygwin has not yet been started. When you do
start it in a later step, this directory must be mounted...unless either:

(1) Cygwin was already running before you began. 'INSTALL' doesn't contemplate
that possibility, because it's written for naive users who can't be running
Cygwin already because they haven't installed it yet. Conceivably, I suppose,
you might have set up your VM to boot with a daemon running via 'cygserver';
but I think Cygwin's 'setup*.exe' installer would give a warning in that case.

(2) Writing to /etc/fstab before starting Cygwin no longer works--which would
be startling.

> GC> [...] Does your
> GC> /etc/fstab offer any clue? Mine says:
> GC>   C:/cache_for_lmi          /cache_for_lmi lmi_specific binary,user 0 0
> 
>  Hmm, I don't have anything in /etc/fstab:
> 
>       % cat /etc/fstab
>       # For a description of the file format, see the Users Guide
>       # http://cygwin.com/cygwin-ug-net/using.html#mount-table
> 
>       # This is default anyway:
>       none /cygdrive cygdrive binary,posix=0,user 0 0

Just now, on a spare machine, I removed all traces of Cygwin and lmi
(even removing all 'cygnus' and 'cygwin' registry entries), and then
followed the directions in lmi's 'INSTALL'. Here's what I saw when I
first opened a 'bash' terminal and examined mounts and /etc/fstab :

Copying skeleton files.
These files are for the users to personalise their cygwin experience.

They will never be overwritten nor automatically updated.

'./.bashrc' -> '/home/Snowy//.bashrc'
'./.bash_profile' -> '/home/Snowy//.bash_profile'
'./.inputrc' -> '/home/Snowy//.inputrc'
'./.profile' -> '/home/Snowy//.profile'

address@hidden ~
$ cat /etc/fstab
# For a description of the file format, see the Users Guide
# http://cygwin.com/cygwin-ug-net/using.html#mount-table

# This is default anyway:
none /cygdrive cygdrive binary,posix=0,user 0 0
#
C:/opt/lmi/MinGW-20090203 /MinGW_        lmi_specific binary,user 0 0
C:/opt/lmi                /opt/lmi       lmi_specific binary,user 0 0
C:/lmi                    /lmi           lmi_specific binary,user 0 0
C:/cache_for_lmi          /cache_for_lmi lmi_specific binary,user 0 0

address@hidden ~
$ mount
C:/opt/lmi/MinGW-20090203 on /MinGW_ type ntfs (binary,user)
C:/cygwin-lmi/bin on /usr/bin type ntfs (binary,auto)
C:/cygwin-lmi/lib on /usr/lib type ntfs (binary,auto)
C:/cache_for_lmi on /cache_for_lmi type ntfs (binary,user)
C:/cygwin-lmi on / type ntfs (binary,auto)
C:/opt/lmi on /opt/lmi type ntfs (binary,user)
C:/lmi on /lmi type ntfs (binary,user)
C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
D: on /cygdrive/d type iso9660 (binary,posix=0,user,noumount,auto)

address@hidden ~

So this certainly seems to be where things go awry. My /etc/fstab
is not like yours. Could it be that you had an old /etc/fstab
left over from some previous Cygwin installation? Or that, on
whatever msw version you're using, but not on my msw-xp-sp3,
/etc/fstab is locked or protected from modification, e.g. by some
kind of ACL stuff? (I wouldn't expect msw to consider /etc/fstab
a crucial file deserving of extra protection.)

> This is suspicious as install_cygwin.bat clearly writes to this file. But
> somehow -- even though I received "Cygwin installation seems to have
> succeeded" message at the end -- this didn't seem to have had any effect.
> Perhaps it's another change in Cygwin 1.7?

It can't be a change in the initial release of Cygwin 1.7, because I
rewrote (and tested) 'install_cygwin.bat' specifically for that release.
Putting mount points in /etc/fstab was one of the most important changes
in 1.7, and I see no suggestion that this has changed--either in the FAQ:
  https://cygwin.com/cygwin-ug-net/using.html#mount-table
or on the Cygwin mailing list, which I always read.

I suppose it's remotely possible that you somehow have old Cygwin 1.5
mount points in your registry:
  https://cygwin.com/faq/faq.html#faq.setup.upgrade-mountpoints
| When you upgrade an existing older Cygwin installation to Cygwin 1.7,
| your old system mount points (stored in the HKEY_LOCAL_MACHINE branch
| of your registry) are read by a script and the /etc/fstab file is
| generated from these entries.

>  FWIW I do have cache_for_lmi mounted on c:/cache_for_lmi:
> 
>       % mount
>       C:/opt/lmi/MinGW-20090203 on /MinGW_ type ntfs (binary,user)
>       C:/cygwin-lmi/bin on /usr/bin type ntfs (binary,auto)
>       C:/cygwin-lmi/lib on /usr/lib type ntfs (binary,auto)
>       C:/cache_for_lmi on /cache_for_lmi type ntfs (binary,user)

I see:
        C:/cache_for_lmi on /cache_for_lmi type ntfs (binary,user)
when I run the same command here. The result is exactly identical.

>       C:/cygwin-lmi on / type ntfs (binary,auto)
>       C:/opt/lmi on /opt/lmi type ntfs (binary,user)
>       C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> 
> GC> Do you have a directory (or other file) such as
> GC>   C:\cygwin-lmi\cache_for_lmi
> GC> in the Cygwin directory? That could cause confusion.
> 
>  Yes, there is indeed such directory and, more interestingly, it has
> downloads subdirectory which is empty:
> 
>       % ls -AR /cygdrive/c/cygwin-lmi/cache_for_lmi
>       /cygdrive/c/cygwin-lmi/cache_for_lmi:
>       downloads
> 
>       /cygdrive/c/cygwin-lmi/cache_for_lmi/downloads:
> 
> 
>  So I finally think I see what's going on here:
> 
> 0. For some unknown but not really unexpected reason

My expectations are more, well, exigent. Unfortunately, they aren't
fulfilled on your machine. But I'm really surprised by this: it just
doesn't seem possible to me.

>    , Cygwin installation
>    script doesn't mount /cache_for_lmi on C:/cache_for_lmi. It is still
>    unknown why /etc/fstab was not updated, but even if it were, I don't
>    see how adding this line to /etc/fstab would actually mount it, there
>    should still be the actual "mount /cache_for_lmi" command somewhere
>    and it just doesn't seem to be present anywhere.

As far as I understand the Cygwin 1.7 documentation cited above,
directories can be mounted by writing /etc/fstab entries, and it is
not necessary to issue an explicit 'mount' command in addition.

But in your case /etc/fstab was not updated. That seems to be a
single point of failure that can explain all these woes; if so,
then that's the failure that we would ideally find some way to
debug, right? If that's not possible, then I guess we have to
work around that failure downstream...

> 1. "mkdir --parent /cache_for_lmi/downloads" in the beginning of
>    install_msw.sh created C:/cygwin-lmi/cache_for_lmi/downloads
>    because /cache_for_lmi hadn't been mounted on C:/cache_for_lmi initially.
> 
>    And this is indeed confirmed by this snippet from the beginning of the
>    log:
> 
>       mount
>       C:/cygwin-lmi/bin on /usr/bin type ntfs (binary,auto)
>       C:/cygwin-lmi/lib on /usr/lib type ntfs (binary,auto)
>       C:/cygwin-lmi on / type ntfs (binary,auto)
>       C: on /cygdrive/c type ntfs (binary,posix=0,user,noumount,auto)
> 
> 2. Then install_msw.sh proceeded to mount /cache_for_lmi, from the log:
> 
>       mount --force "C:/cache_for_lmi" "/cache_for_lmi"
> 
> 3. Then "make -f install_mingw.make" failed because now
>    /cache_for_lmi/downloads maps to (non-existent) C:/cache_for_lmi/downloads
>    directory.
> 
> 4. Then "make -f install_mpatrol.make" created that directory and
>    subsequent downloads were successful, but building everything failed
>    because of lack of the compiler.
> 
> 
>  Sorry for failing to understand it on my own and thanks for your help with
> debugging it.
> 
>  There are several ways to fix this, the two most direct ones I see are:
> 
> (A) Move "mkdir --parents /cache_for_lmi/downloads" command below the
>     mount commands in install_msw.sh. This is the simplest and most minimal
>     solution.

I can't reproduce the problem, so can you tell me whether the following
patch suffices? If so, I'll be glad to apply it. It seems logical:
in the unhoped-for case where this line:
  mount --force "C:/cache_for_lmi" "/cache_for_lmi"
actually makes a difference, this line:
  mkdir --parents /cache_for_lmi/downloads
probably ought to *follow* it.

Index: install_msw.sh
===================================================================
--- install_msw.sh      (revision 5952)
+++ install_msw.sh      (working copy)
@@ -32,15 +32,9 @@
 # To remove lmi prior to reinstalling with this script:
 #
 # rm --force --recursive /opt/lmi
-#
-# Downloaded archives are kept in /cache_for_lmi/downloads/ because
-# they are costly to download and some host might be temporarily
-# unavailable.

 date -u +'%Y%m%dT%H%MZ'

-mkdir --parents /cache_for_lmi/downloads
-
 mount

 # Establish mounts carefully.
@@ -115,6 +109,12 @@
   || echo -e "Replacing former cache mount:\n  $restore_cache_mount" >/dev/tty
 mount --force "C:/cache_for_lmi" "/cache_for_lmi"

+# Downloaded archives are kept in /cache_for_lmi/downloads/ because
+# they are costly to download and some host might be temporarily
+# unavailable.
+
+mkdir --parents /cache_for_lmi/downloads
+
 mount

 md5sum $0