Re: [Gnu-arch-users] Potential flaw in patch-log pruning in proposal

[Aaron Bentley]

Thomas Lord wrote:
>     > From: Aaron Bentley <address@hidden>
>
>     > One of the features of Arch I really like is the history-sensitive 
>     > merging.  This process breaks history-sensitive merge commands.
>
> That is a complete misapprehension.

No, that is totally accurate.  The process that Matthew Dempsky used to 
commit patch-7 breaks all conceivable history-based merge commands, and 
requires a human to determine its origins.

Now, if we assume the =merges idea is implemented, it may or may not be 
possible to fix the commands, but it breaks the commands we have right 
now, today.

> Rather, this process relies on history-sensitive merges that aren't
> yet built-in, but which are minor variations on merge commands which
> are built in.

Please build in the new variations before removing support for what we 
already have.  Please don't implement a draft proposal before we've 
finished discussing it.  Or is it no longer a draft?  Either way, it 
never had =merges.

> Let's suppose that I have a mainline, M, and maintain =merges and
> =pending.   You have an experimental branch X.
>
> X is maintained by:
>
>   1. Periodically star-merging from M

My current X is not maintained using star-merge, because 
address@hidden/tla--devo--1.3 occasionally cherry-picked changes 
from me.  Even if, in the future, I submitted versions, those versions 
would contain changes that originated in X.  So I do not believe this is 
a useful scenario to explore.

>   2. Periodically star-merging from each version in the
>      set difference of =cherries and =merges
>
>   3. Arbitrary, other, X-specific changes
>
> You can star-merge from M freely.  That's the only thing that should
> ever change '=merges' (or we can make '=merges.$VERSION') in your
> tree.

I cannot star-merge from M freely, because I can't detect when M merges 
changes that originated in X.

Actually, I can, but to accomplish that, I have to keep a record of all 
the submit versions that were derived from X in X.  And then I need to 
determine whether a changeset adds one of those to X before applying it.

Removing the patchlogs doesn't just remove patchlog files, it also 
removes their entries from the new-patches header.  So there is no way 
to determine whether a changeset adds patches that have already been 
merged without downloading the changesets.

> No intentionally-recorded history is lost in this set-up: your X
> version is perfectly fine.

It was intentional when I committed those logs.

There's all kinds of useful data in the patchlogs.  For example, want to 
know who and what added the following from line from pfs-sftp.c?  Oh 
come on, humour me.

safe_printfmt(2, "Error creating directory %s in path %s: File 
exists\n", path, pfs->cwd);

$fai revisions --modified pfs-sftp.c:1127 -scD
address@hidden/tla--devo--1.2--patch-56
    Tom Lord <address@hidden>
    2004-01-11 19:45:51 EST
    nicer error messages at pfs layer (abently)

We can even trace into the new-patches header.  Fai did that, but was 
deceived, because it looks for exact matches, and you changed some of 
the formatting before committing.

>   At the same time, patch-tree log sizes in the mainline can be kept to 
a bounded size.

I believe we can strike a balance between unbounded growth and no 
historical data; I think the patchlog *contents* don't belong in the 
tree, but an index of merged patches *does*.  Try this:

for version in $(tla log-versions); do tla logs -f $version; done

The output for address@hidden/tla--devo--1.3--patch-28 is a mere 
14k.  We could go for years with that kind of accumulation.  And if we 
can use patchlevels instead of fq-revisions, that drops to about 2.3k.

All we have to do is bump the working tree format and modify the 
functions that interact with tree logs.  We can cache the 
commonly-requested logs, or maybe keep some logs (e.g. the tree-version 
logs) in the tree and use the library/archive logs for the rest.

That way, no commands would break, and we'd avoid the patchlog 
explosion.  Keep the list sorted, and access time cannot be a serious 
concern.

If that were in place now, you wouldn't be thinking about pruning, 
right?  Can you hold off until we do get something like that in place?

Aaron
--
Aaron Bentley
Director of Technology
Panometrics, Inc.