[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: better buffer size for copy

From: Phillip Susi
Subject: Re: better buffer size for copy
Date: Sat, 19 Nov 2005 10:49:07 -0500
User-agent: Mozilla Thunderbird 1.0.7 (X11/20051010)

I don't see why the filesystem's cluster size should have a thing to do with the buffer size used to copy files. For optimal performance, the larger the buffer, the better. Diminishing returns applies of course, so at some point the increase in buffer size results in little to no further increase in performance, so that's the size you should use. I believe that the optimal size is about 64 KB.

Robert Latham wrote:

(README says to ping if there's not been an ack of a patch after two
weeks.  here i go)

This patch to today's (18 Nov 2005) coreutils CVS makes copy.c
consider both the source and destination blocksize when computing
buf_size.  With this patch, src/copy.c will use the LCM of the soruce
and destination block sizes.  As Paul suggested, I used the buffer_lcm
routine from diffutils.
For what it's worth, this patch does not introduce any regressions
into the coreutils testsuite.

When copying from a remote filesystem with a block size of 4MB to a
filesystem with a 4k blocksize, the copy is *very* slow.  Going from a
filesystem with 4k blocks to a filesystem with 4MB blocks is much
faster.  With this patch, both operations are equally performant.

I went ahead and added a ChangeLog entry as well.
Thanks.  I'll be more than happy to incorporate any suggestions or


reply via email to

[Prev in Thread] Current Thread [Next in Thread]