Re: Shred: Add recursion operations [PATCH]

[Amr Ali]

On 12/09/2011 08:24 PM, Eric Blake wrote:
> On 12/09/2011 10:52 AM, Amr Ali wrote:
>> Hi Eric,
> 
> Please don't top-post on technical lists.
> 
>>
>> I completely agree about the philosophy. But ...
>>
>> 1) I've seen a comment at shred.c:26 that indicates adding -R/-r/--recursive 
>> as
>> a TODO.
> 
> Hmm, we probably ought to nuke that comment instead, unless there are
> other implementations of shred that already provide -r.
> 
>> 2) Adding recursion to shred is not bloat, it is in fact enabling shred to do
>> it's job, and to do it *well*.
>> 3) On the command line, the overhead of first, having to type down all of the
>> find parameters, and second, the overhead of executing shred, quite 
>> literally,
>> on every single file is a waste of both time and resources. Adding a simple 
>> `-r`
>> flag, however, enhances usability dramatically compared to the find(1) 
>> solution.
> 
> You misunderstand 'find -exec shred {} +', which executes as few shreds
> as necessary to fit within command-line limits.  Shred operates on
> multiple files per pass, and I don't see how teaching shred the ability
> to recurse adds any noticeable speed over using find to do the
> recursion.  The bulk of your time is spent not in recursion, but in
> shredding, and using -exec + to avoid fork()ing one shred per file
> should put the overhead in the noise.  Which leaves shred to do one
> thing well - shredding individual files.
> 
You do have a point. Indeed the quantum taken by shred processing files far out
weights that of the fork-per-directory CPU utilization. Despite that this can be
argued as it largely depends on the directory structure depth and spread, but
it's arguable too that the likelihood of a complex directory structure layout
lies in the 10th percentile.
>> 4) It became more than obvious that a lot of users *do* want such ability if 
>> you
>> will, given the amount of postings online that talks/addresses the shred
>> recursion issue.
> 
> You didn't post any such links to support your claim.  Meanwhile, I can
> point to the coreutils FAQ:
> https://www.gnu.org/software/coreutils/faq/#Why-don_0027t-the-utilities-have-built-in-directory-recursion_003f
> and state definitively that I have used that FAQ less than 5% of the
> time where the FAQ already provided my answer (my most frequent uses are
> about 'sort not working' and 'date not working').
> 
As for the links, a simple Google query like "shred+no+recurse" should show that
people were either disappointed or looking for a workaround/alternative that
works recursively on a directory. e.g. ..

http://www.anchor.com.au/hosting/dedicated/WipingDeletingAndShreddingCustomerDataBeforeReleasingHardware
http://www.linuxforums.org/forum/miscellaneous/46693-how-shred-entire-directory-tree.html
http://ubuntuforums.org/archive/index.php/t-1082954.html

These mainly show that there is a "usability" issue since 2005 and probably a
little earlier than that, *and* it still remain!

Point really is, I'd naturally expect shred to have a recursive mode, why go
against nature? some users always appreciate a consistent way of doing things.

Apparently this is a far more cultured and dense mailing list than expected.
Silly me, apologies for not taking the necessary steps before going into PATCH
mode. I should have read that before having you to explain it again.
>> 5) If `find` truly is made to enable recursion for all programs, why 
>> recursion
>> is available for chmod or chgrp, or the plethora of other tools?
> 
> 'chmod' and 'chgrp' have recursion because they affect the recursion
> itself. I can dream up recursion situations where find cannot operate
> unless you change permissions as you go, but the only way to change
> permissions as you go is to do one chmod invocation per name, which is
> inefficient.
> 
> 'du' and 'ls' have recursion because they output a total - if you were
> to break things up by command-line length limits, you would have
> multiple totals.
> 
> 'rm' has recursion even though 'find -delete' does the same, because
> 'find -delete' is a GNU extension not required by the standards.
> 
> The goal is that if 'find -exec +' can do your task, because no
> permissions are being modified by your task and your task has no output
> that would be adversely broken up by multiple max-command-line-length
> invocations, then your task doesn't need builtin recursion.  'shred'
> fits this goal, 'chmod', 'ls', and 'rm' do not.
> 
The act of outputting a total, modifying permissions, or standards that focus on
low level technical particularities is irrelevant. It's the process of working
on multiple items to arrive at a quantifiable result that illustrates the
purposefulness of recursion.

Purpose is what drives my modifications towards shred. If you do work on
multiple files (and that *does* have a high probability), with a simple command
line regular expression (e.g. './*') you are in fact recursing over files of the
first level of depth of the current directory node, which conditionally, having
it recurse on all directory nodes on multiple levels is only an expected
behavior or for the very least having the ability to switch between modes is
always thoughtful. (If you have a robot arm, and it moves only in 2D-space,
moving in 3D-space is not bloat or out of line, it's only an expected behavior,
if unavailable then it is an expected upgrade)
> Adding recursion to coreutils is NOT done lightly.  You can still manage
> to convince me that 'shred' fits one of the exceptions I listed above
> for needing built-in recursion, but the burden is on you to do so, and
> so far you have not convinced me.
Before taking a dive at coreutils source code, which was my very first time, I
always enjoyed just using the *functionalities* it does provide, I knew how they
do work at very low level details, but this is not the drive behind using them.
It's simply the desired quantifiable result. The low level technical
particularities are almost always expected to be abstracted away from usage.
> 
> Finally, if you DO manage to convince me, then we would need copyright
> assignment before your patch could be applied.
> 
Should I use `assign.change.manual`? if so which email I should be sending this
to, after signing it? That of course if I did convince you.

-- 
Amr Ali

signature.asc
Description: OpenPGP digital signature