[coreutils] Re: cp command performance

[Bob Proulx]

Hemant Rumde wrote:
> I do not log any bug for cp command.

In that case I will close the bug report that you have opened.

Let's have the discussion on the discussion mailing list
address@hidden.  That is the more appropriate place.  I have set
the mail headers to direct discussion there but if your mailer doesn't
comply please manually redirect it.

> In our company, we copy huge Cobol files before processing
> data. This is to rollback our data files.  Suppose A1 is my huge
> file of 60GB and A1.bk is its backup file, before we process (
> change ) data into A1. Then which of our method would be faster?
> 
> 1. Method-1 ( A1.bk exists ) 
>     $ cp  A1 A1.bk 
> 
> 2. Method-2  
>     $ rm -f A1.bk 
>     $ cp A1 A1.bk 
> 
> 3. Method-3 
>     $ cp --remove-destination A1 A1,bk 

All three of those should be virtually the same, especially the last
two.  But benchmarking it is always good.  I created a 10G test file
using dd and copied it once to set up the test and then performed the
following operations on a ext3 filesystem.

  $ time cp testdata testdata.bak
  real    3m34.435s
  user    0m0.108s
  sys     0m30.950s

  $ time ( rm -f testdata.bak ; cp testdata testdata.bak )
  real    3m27.941s
  user    0m0.092s
  sys     0m30.914s

  $ time cp --remove-destination testdata testdata.bak
  real    3m36.931s
  user    0m0.068s
  sys     0m30.862s

As you can see the times for all three operations are with limits of
being exactly the same.

> This operation is very simple. But our operators tell, in some cases cp
> takes longer time. How can we reduce copying time?

I do not doubt that there will be differences in times consumed for
just the raw command.  With such a large file I think this will be
dependent upon outside influences.  Such as what filesystem you are
using for the copy and how much ram you have available for buffer
cache and whether extraneous sync and fsync calls are happening at the
same time and so forth.  I could send for-examples but I don't want to
send you off on in the wrong direction and so will resist.

Bob