[Gnu-arch-users] Re: Caching (was: GNU Arch review - am I accurate?)

[Stefan Monnier]

>> It could be done by tla as well.
>> Admittedly, OSes should have the notion of "disposable files" so you could
>> use all the disk space with revlibs and they'd be deleted by the OS when
>> more diskspace is needed.  But in the mean time, it'd be nice to have tla
>> do the pruning itself.

> Yes, you can. However, you usualy need to get it pruned when you DON'T
> work with tla...

I don't know about "usually".  I do know that it would be convenient if tla
did the pruning itself without me having to setup some cron job.
But yes, a cron job could be helpful as well.

I think it's important that people can just use tla with a strict minimum of
setup work and get reasonable performance and behavior.  Currently, you need
to give my-id, my-revision-library, potentially setup a cron job which you
have to download separately, mark your revlib as greedy (and maybe as sparse
as well), ...

>> > For offline work, it may never be automatic, because you must tell it
>> > what you want to have cached for offline work. Archive mirroring will do
>> > the trick.
>> There's no reason why it can't be as automatic as revlib: just cache
>> whatever has been fetched from the network.
> It's not the same. It would have to be: Before every operation on the
> mirror, do an archive-mirror.

I am *not* talking about a mirror but about a cache.  I am talking about
caching patches and/or whatever other data might be fetched from
the network.  Caches are transparent, mirrors are not.

>> >> Mirrors are not at all automatic.  A far cry from a "cache".
>> > They can be made automatic for backup purpose by putting push-mirror in
>> > a hook.
>> Huh?  What hook?  What mirror?  Last I tried you can't modify a mirror, so
>> there's no hook to use there.  As for the main archive, you generally don't
>> have write access to it, so you can't add a hook there.
>> But in any case, the problem is mostly the setting up of mirrors which no
>> amount of push-mirror will automate for you.  Revlibs are much better here.
>> Furthermore, mirrors only cache "the whole thing or nothing".

> For *BACKUP* purposes, you commit to master and push-mirror to the
> backup. There IS a hook.

Sorry, I did not notice you switched to talking about backups.
I'm only talking about caches.

>> > For offline work they can't really be automatic, because you
>> > need to say on what you want to work. And for caching revision libraries
>> > are usualy better.
>> A good way to say "I need this data" is to access it.  So having
>> transparent caching makes perfect sense for offline work as well.
>> I do that all the time with revlibs: I never explicitly add something to
>> the revlib, instead I just do `tla changes' to make sure the base revision
>> is in the revlib.
> You must also say "and don't dare to prune them".

You should be able to expect that the cache will not prune the things
you've accessed recently.  When I `get address@hidden/foo--bar--baz' tla might
end up fetching data from several archives and I do not want to have to
know which ones (it's bad enough that I need to register them) so as to
know what to mirror locally.

But yes, if you want to be 100% sure that you'll be able to grovel through
all the revs of some project, you should probably setup a local mirror
because a cache might not be the right answer for that.

>> >> And also offline work.  That's closer but:
>> >> - You have to set it up manually (which is not that bad since it's only
>> >> done once, but still: it should be setup automatically with a sensible
>> >> default).
>> > It's a bit hard -- you have to say where they should be put.
>> What's hard about choosing a sensible default?
> No, I don't like that.  I want to know, what it's doing.

Don't worry, it'll be documented and it can even be mentioned explicitly by
tla the first time it's run.

> However, it might be nice if tla, when run for the first time by a new
> user did:

> 1) Ask for user-id (trying to find real name and e-mail by standart
>    means and provide that as a default).
> 2) Ask for location of revision library and set it up (you could answer
>    "Not now" to which it would replay with command you need to type to
>    do it later).
> 3) Ask for location and name for default archive (the same argument with
>    "Not now" applies here).

I don't like it when a command that normally works in batch mode suddenly
expects user input just because it happens to be the first time it's run.
(imagine, you open an Arch-controlled file in someone else's directory,
Emacs runs `tla inventory' to figure out whether it's source or not and gets
all confused when tla hangs waiting for some answer because you've never
used tla yourself yet).

But adding a `tla setup' command might make sense.  I still think that the
revlib could be setup by default without any user involvement.

>> >> - It does not cache patches.
>> > Sure it DOES.
>> 
>> My definition of caching is "transparently keeping copies nearby".
>> Patches are never cached in this sense.  You can manually copy them nearby,
>> but that's just mirroring, not caching.

> Revision library contains **full unpacked changeset** for each revision.
> Just look for directories named ,,patch-set in the library.

Ha!  Good point.  I didn't know about that one.  Sadly it only keeps the
patch corresponding to the revision, so the patch can never be used to
forward-build a missing revision.  Are those patches ever used by tla?
Will they be used by the upcoming backward-build stuff?  How about keeping
patch-15 when greedily adding revision patch-16 (built from rev patch-14) to
a sparse revlib?

>> >> - It does not cache some other meta data (not sure which, but when I do
>> >> `tla build-config ...tla-1.2' tla somehow needs to access the network
>> >> and check some patches's sigs even when all the trees's revisions are
>> >> in the revlib).
>> > Does the config only contain _revision_, or does it contain _versions_
>> > too?
>> I wouldn't know.  I just want to rebuild Tom's tla-1.2 and know nothing of
>> how he organizes his thing.
> Then please don't jump on conclusions with "patches' sigs".

That's right, it checks some sigs, and maybe they're not patch-sigs.

>> > In the second case, it needs to look which is the latest revision...
>> Yes, although I don't see why it should need to check any signature since
>> the revision is ends up using is already in the revlib.  I thought
>> signatures where only added to patches, not to "the current list of patches
>> of this version".  But I admit ignorance here.
> What's your evidence that it downloads the _sigs_?

I have no such evidence, but I do have evidence that it checks some sigs
(since I get gpg's messages).


        Stefan