RE: Rm performance issue

[Philip Rowlands]

[ re-adding bug-coreutils]

On Wed, 26 Sep 2007, Ken Naim wrote:

> I created a test case similar to my nightly job which removes 130 or 
> so 5gb files. The apps* files are identical to files I remove nightly 
> and are 4.3 gb in size, the ctx files are 20mb, and the single letter 
> files are 0 byte files. Based on the strace there is a correlation 
> between file size and unlink time. 4.3gb takes 10 seconds, 20mb take 
> less than .1 seconds and 0 byte files take no time.

> unlink("apps_ts_tx_data_2.270.632954231") = 0 <10.418224>
> unlink("apps_ts_tx_data_2.271.632954231") = 0 <10.691083>
> unlink("ctxd.367.632955010")            = 0 <0.051140>
> unlink("ctxd.367.632955011")            = 0 <0.078666>
> unlink("d")                             = 0 <0.000059>
> unlink("e")                             = 0 <0.000050>

As I thought, rm is limited by the speed of the underlying I/O. I'd 
suggest some performance tuning for your filesystem and SAN, but that's 
very dependent on your current setup. unlink shouldn't cause much I/O 
compared to other read/write operations, so I'm surprised you only 
noticed issues with rm.


Cheers,
Phil