Re: [Gnu-arch-users] tla1.2 on cygwin

[Tom Lord]

    > From: John F Meinel Jr <address@hidden>

    > I am obviously very interested having you maintain this.

Well, I can't _maintain_ it -- just merge it and try not to break it
too badly with future changes.    This isn't obviously losing -- GNU
Emacs development worked (perhaps still does?) that way for many years
(although I'd hope to avoid quite so many #ifdefs).

    > One problem I ran into with the newest version is that it compresses the 
    > .arch-ids directory, so then when you do a tla tree-lint, it doesn't 
    > recognize that a sub-directory is part of the archive. Which is odd, 
    > considering that most patches will still apply. It does complain about 
    > certain patches now, which I don't understand.

    > I'm not sure if some of this problem is coming from when I switched from 
    > explicit tagging to tagline tagging.

    > Either way, the latest version won't actually let me checkout my 
    > repository anymore, so I've reverted back to the old one.

Can you briefly describe for me how you've implemented path
compression?   I'll tell you how I _think_ it should work -- this is
basically what you did, right?

I _think_ that path compression should happen entirely at the VU
layer -- it should be transparent to most of arch.   VU let's you
define something called a "namespace handler" so that if a filename is
passed to any of:

                 access
                 chdir
                 chmod
                 chown
                 chroot
                 link
                 lstat
      vu_        mkdir
                 open
                 opendir
                 readlink
                 rename
                 rmdir
                 stat
                 symlink
                 unlink
                 utime

the call can be intercepted and the path modified.

Not _all_ paths should be compressed.  For example, you want to
compress paths under {arch} but not paths to source files in the
tree.   You might want to compress files under some of the temp
directories created by arch, etc.

And, compression doesn't necessarilly apply to an entire path.  For
example, if you have a path for the form:

        <before>/{arch}/<after>

then the <after> part needs to be compressed but not the <before>
part.

But the namespace handler mechanism has enough flexibility to handle
that.    Roughly speaking, it should just require a bunch of lines in
main() registering various regexps and rules for various kinds of
paths -- and that can be driven by a big table (making it easy to
modify if we add new paths to arch in the future).

There's a slightly tricky bit.  vu_opendir has to be prepared to
compress the path it is given as an argument but it goes further than
that.   If the path it's given is the root of a subtree which contains
compressed paths (or is somewhere in that subtree) then the VU
_descriptor_handler_ that gets used must have special handlers for:


                     readdir
        vu_          closedir


so that vu_readdir returns uncompressed names.

And there's another (somewhat nasty) tricky bit.  Tla uses chdir and
fchdir internally.   Having chdir'ed to some directory, it uses paths
relative to that directory.   For example, tla might chdir into a
.arch-ids directory and then opendir "." (I don't know if it literally
does that specific example but it can do that sort of thing.)

The means that you have to keep track of cwd in the VU code and that,
in turn, means that you need to provide a namespace handler for _all_
path names -- even those that are uncompressed.   For example, if a
caller first does:

        fd = vu_open (&errn, ".", O_RDONLY, 0);

and much later does:

        vu_fchdir (&errn, fd);

you have to wind up back in the right place.   And, being back in that
right place, a call like:

        fd = vu_open (&errn, "foo.c.id", O_RDONLY, 0);

has to know whether or not compression applies to that path.

Related to that, there's one final (nasty) tricky bit (that I'm aware
of:-):

Completely outside of VU, in ./src/hackerlab/fs/cwd.c, libhackerlab 
uses a naked call to `getcwd'.   This obviously won't quite do: the VU
abstraction needs to be extended in a way that permits getcwd to be
overridden (so that programs see the uncompressed path).

I have to think for a little while about the best way to do getcwd
handling but I'll leave that for now to find out how all of this
compares to what you've actually done.

So when you say:

    > One problem I ran into with the newest version is that it
    > compresses the .arch-ids directory, so then when you do a tla
    > tree-lint, it doesn't recognize that a sub-directory is part of
    > the archive.

If the approach you took is like what I described above, the fact that
.arch-id paths are compressed on creation but not recognized by
tree-lint _suggests_ that tree-lint is making a (vu) system call that
isn't inverting the compression.

As a wild guess:  perhaps you missed the need for special readdir
handling or perhaps you missed one of the system calls in the long
list above (stat, lstat, and access are likely suspects).

Alternatively: all the system calls are properly covered but tree-lint
is using a path that is different in form than the path form used to
create the file (but equivalent in meaning, in unix).  One of your
namespace patterns fails to match the path form used by inventory.

Alternatively: something in the details of cwd-handling is wrong,
inventory uses chdir and fchdir heavily, and so uncompression isn't
happening quite right.

-t