[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#51427] [PATCH] nix: libstore: Do not remove unused links when delet
[bug#51427] [PATCH] nix: libstore: Do not remove unused links when deleting specific items.
Mon, 08 Nov 2021 23:11:22 -0500
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
Ludovic Courtès <email@example.com> writes:
> Hi Maxim and all,
> Maxim Cournoyer <firstname.lastname@example.org> skribis:
>> Ludovic Courtès <email@example.com> writes:
>>> You seem to be proposing to remove ‘-D’ altogether. I agree it has the
>>> shortcomings you write, but I think it’s occasionally useful
>>> My proposal would be either the status quo, or removing just the one
>>> link that matters from /gnu/store/.links upon ‘-D’.
>> The second proposal makes sense.
> Maybe that proposal is bogus though because you’d need to know the hash
> of the files being removed, which means reading them…
>> I didn't care about freeing space, as my use case was getting around
>> corrupting an item in my store due to
>> https://issues.guix.gnu.org/51400, which the patch proposed here
>> allowed me to do without wasting hours of cleaning up links (nearly 1
>> GiB of store on spinning drives).
> The ideal solution as zimoun writes would be to address
Seems there's some improvement ready, but which needs more
testing/measurements? I'd suggest simply invoking GNU sort; if it has
many pages of program for doing what it does, it's probably doing
something fancier/faster than we can (are ready to) emulate -- for free!
> Perhaps that phase needs to be implemented using a different strategy,
> say an sqlite database that records the current link count (hoping that
> ‘SELECT * FROM links WHERE NLINKS = 1’ would be faster than traversing
> all of ‘.links’) as well as a mapping from store item to file hashes.
Hmm. I'll need to dive in the problem a bit more before I can comment
> BTW, those using Btrfs can probably use ‘--disable-deduplication’ and be
> done with it.
I erroneously used to think that Btrfs could do live deduplication, but
it doesn't. There are external tools to do out of band / batch
deduplication though ; so if they perform better than the guix daemon's
own dedup, perhaps we could document this way out for our Btrfs users.