[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: better buffer size for copy

From: Robert Latham
Subject: Re: better buffer size for copy
Date: Tue, 22 Nov 2005 11:14:03 -0600
User-agent: Mutt/1.5.9i

On Mon, Nov 21, 2005 at 12:45:40AM -0500, Phillip Susi wrote:
> If we are talking about the conventional blocking cached read,
> followed by a blocking cached write, then I think you will find that
> using a buffer size of several pages ( say 32 or 64 KB ) will be
> MUCH more efficient than 1024 bytes ( the typical local filesystem
> block size ), so using st_blksize for the size of the read/write
> buffer is not good.  I think you may be ascribing meaning to
> st_blksize that is not there. 

I mean no offense cutting out most of your points.  You describe great
ways to achieve high I/O rates for anyone writing a custom file mover.
I shouldn't have mentioned network file systems.  It's a distraction
from the real point of my patch: cp(1) should consider both the source
and the destination st_blksize.

All I expect from st_blksize is what the stat(2)
manpage suggests:

       The value st_blocks gives the size of  the  file  in  512-byte
       blocks.  (This  may  be  smaller than st_size/512 e.g. when the
       file has holes.) The value st_blksize gives the "preferred"
       blocksize for efficient file system  I/O.  (Writing to a file
       in smaller chunks may cause an inefficient

All I really want is for cp(1) to do the right thing no matter what
the soruce or destination st_blksize value might be. 

In copying from a 4k blocksize file sytem to a 64k blocksize
filesystem, cp(1) will perform well, as it is using a 64k buffer.  

In copying *from* that 64k blocksize filesystem *to* a 4k blocksize
filesytem, cp(1) will not perform as well: it's using a 4k buffer and
so reading from the source filesystem in less-than-ideal chunks.

Thanks again for taking the time to respond.  I hope I have made the
intent of my patch more clear. 


Rob Latham
Mathematics and Computer Science Division    A215 0178 EA2D B059 8CDF
Argonne National Labs, IL USA                B29D F333 664A 4280 315B

reply via email to

[Prev in Thread] Current Thread [Next in Thread]