Re: CP Proposal

[Paul Eggert]

"Thorsten W. Schmidt" <address@hidden> writes:

>> Why bother with a user-specified flag?  Can't cp always invoke
>> posix_fadvise with the POSIX_FADV_SEQUENTIAL | POSIX_FADV_NOREUSE
>> option?
>
> Yes, that is possible. But what to do if a user knows, that he will reuse
> the copied file(s)? Or the source file(s)? Then it would be good to keep
> them cached. I guess a good heuristic would be: Turn the flag allways on
> except if a single small file is copied.

But then we would need to worry about what "small" means....
But see below.

> Of course the flag needs to be turned on for reading AND for writing!

Yes.

>> Have you tried benchmarking this on a real GNU/Linux system?  There's
>> no point doing it if it doesn't help much.
>
> Yes, I did now and found out that it's not working as I thought

OK, in that case let's drop the posix_fadvise idea, at least for now.
There's no point hassling with these flags if they don't really help.
Perhaps once Linux gets more mature we can revisit this.

> but I found another solution: O_DIRECT that is working as
> expected.

Ah.  OK, but that is another matter.  cp probably should not use
O_DIRECT by default since it can hurt performance in many cases, but
it might make sense to have an option for that, at least on hosts that
have O_DIRECT.

O_DIRECT is not in POSIX, but O_DSYNC is.  Have you tried O_DSYNC?
O_DSYNC might be more portable.


> Is there a possibility to overwrite the open/fopen method, so that you
> might make a call like "program_to_turn_off_caching program". so that you
> could also use "command_to_turn_off_caching cp * /tmp/". This would have
> the advantage that i can turn off caching for every program I want.
> Candidates are tar, bzip2, gzip, ... or any other software that makes a
> lot of file processing like video editing software. What i mean is
> something similar to the command nice.

That would require cooperation with the C library (and perhaps the
kernel); it's a feature that might be useful, but it's really more
a glibc question.

> PS: See attached the file I used to generate the benchmark.

Interesting.  Could you please email the file and the benchmarks to
bug-coreutils, so that everyone else can see them too?  Thanks.