Re: [bug #17903] cp/mv only read/write 512 byte blocks when filesystem blksize > 4MB

[Tony Ernst]

I believe the larger block sizes are especially beneficial with RAID. 
I'm adding Geoffrey Wehrman to the CC list, as he understands disk I/O 
much better than I do.

I don't think a cap is really necessary.  This test and arbitrary 
limitation were put in to work around a specific problem on hpux.  But 
it has side-effects that reach beyond hpux and the original problem. 
So the test should either be limited to hpux-only, or fixed to get rid 
of the side-effects by making it more specific to the original problem.


On Tue, Oct 03, 2006 at 02:02:58PM -0400, Phillip Susi wrote:
> Why not simply cap the size at 4 MB?  If it is greater than 4 MB just go 
> with 4 MB instead of 512 bytes.  In fact, you might even want to cap it 
> at less than 4 MB, say 1 MB or 512 KB.  I think you will find that any 
> size larger than the 32-128 kb range yields no further performance 
> increase and can even be detrimental due to the increased memory pressure.
> 
> Tony Ernst wrote:
> >Hi Paul,
> >
> >Unfortunately, there isn't really a "largest" legitimate st_blksize 
> >for XFS file systems, or I should say the maximum is whatever fits 
> >into a size_t (64 bits).  It's dependent on the stripe width.  I 
> >talked with an XFS developer who told me that 2GB is not out of the 
> >question today.
> >
> >Now there is also the question of whether or not we really want cp/mv
> >allocating a 2GB buffer, but that's a different issue (and a site with 
> >that kind of disk set-up probably also has plenty of memory).
> >
> >Since the 4MB check was to fix a specific problem on hpux, it seems 
> >like that check should occur inside the "# if defined hpux ..." section.
> >At the very least, since the bogus value returned by hpux is such an 
> >strange number, maybe we could just change:
> >                     && (statbuf).st_blksize <= (1 << 22)) /* 4MB */ \
> >to:
> >                     && (statbuf).st_blksize != 2147421096) \
> >
> >I would be very surprised if 2147421096 was ever a valid st_blksize on 
> >any system/filesystem.  It's not a power of 2, or even a multiple 
> >of 128, 512, etc.
> >
> >% factor 2147421096
> >2147421096: 2 2 2 3 3 17 37 47417
> >
> >Thanks,
> >Tony