bug-coreutils
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: FW: Rm performance issue


From: Jim Meyering
Subject: Re: FW: Rm performance issue
Date: Wed, 26 Sep 2007 18:35:11 +0200

[it's good to reply to the list, so I've Cc'd it]

"Ken Naim" <address@hidden> wrote:
> I created a test case similar to my nightly job which removes 130 or so 5gb
> files. The apps* files are identical to files I remove nightly and are 4.3
> gb in size, the ctx files are 20mb, and the single letter files are 0 byte
> files. Based on the strace there is a correlation between file size and
> unlink time. 4.3gb takes 10 seconds, 20mb take less than .1 seconds and 0
> byte files take no time.
>
> I appreciare your help.
>
> address@hidden foo2]$ strace -T -e trace=file rm -f * >../x.log
> execve("/bin/rm", ["rm", "-f", "apps_ts_tx_data_2.270.632954231",
> "apps_ts_tx_data_2.271.632954231", "apps_ts_tx_data_2.272.632954231",
> "apps_ts_tx_data_2.273.632954231", "apps_ts_tx_data_2.274.632954231",
> "apps_ts_tx_data_2.275.632954271", "apps_ts_tx_data_2.276.632954291",
> "apps_ts_tx_data_2.277.632954313", "apps_ts_tx_data_2.278.632954313",
> "apps_ts_tx_data_2.278.632954333", "apps_ts_tx_data_2.279.632954313",
> "apps_ts_tx_data_2.279.632954353", "apps_ts_tx_data_2.280.632954373",
> "ctxd.367.632955010", ...], [/* 32 vars */]) = 0 <0.000292>
> access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or
> directory) <0.000062>
> open("/etc/ld.so.cache", O_RDONLY)      = 3 <0.000062>
> open("/lib/tls/libc.so.6", O_RDONLY)    = 3 <0.000064>
> open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3 <0.000070>
> unlink("apps_ts_tx_data_2.270.632954231") = 0 <10.418224>
> unlink("apps_ts_tx_data_2.271.632954231") = 0 <10.691083>
> unlink("apps_ts_tx_data_2.272.632954231") = 0 <9.708125>
> unlink("apps_ts_tx_data_2.273.632954231") = 0 <11.170446>
> unlink("apps_ts_tx_data_2.274.632954231") = 0 <10.192923>
> unlink("apps_ts_tx_data_2.275.632954271") = 0 <9.677868>
> unlink("apps_ts_tx_data_2.276.632954291") = 0 <10.157322>
> unlink("apps_ts_tx_data_2.277.632954313") = 0 <10.624669>
> unlink("apps_ts_tx_data_2.278.632954313") = 0 <10.640957>
> unlink("apps_ts_tx_data_2.278.632954333") = 0 <10.649074>
> unlink("apps_ts_tx_data_2.279.632954313") = 0 <9.764071>
> unlink("apps_ts_tx_data_2.279.632954353") = 0 <9.486272>
> unlink("apps_ts_tx_data_2.280.632954373") = 0 <9.312557>

As I said, it looks like something related to your file system.
What type of file system are you using?  "df -T ." will tell you.

> unlink("ctxd.367.632955010")            = 0 <0.051140>
> unlink("ctxd.367.632955011")            = 0 <0.078666>
> unlink("ctxd.367.632955012")            = 0 <0.057871>
> unlink("ctxd.367.632955013")            = 0 <0.051694>
> unlink("ctxd.367.632955014")            = 0 <0.084658>
> unlink("ctxd.367.632955015")            = 0 <0.047987>
> unlink("ctxd.367.632955016")            = 0 <0.049673>
> unlink("ctxd.367.632955017")            = 0 <0.098318>
> unlink("ctxd.367.632955018")            = 0 <0.059037>
> unlink("ctxd.367.632955019")            = 0 <0.049887>
> unlink("ctxd.367.632955020")            = 0 <0.092457>
> unlink("ctxd.367.632955021")            = 0 <0.060425>
> unlink("ctxd.367.632955022")            = 0 <0.045415>
> unlink("ctxd.367.632955023")            = 0 <0.067629>
> unlink("ctxd.367.632955024")            = 0 <0.068119>
> unlink("ctxd.367.632955025")            = 0 <0.044039>
> unlink("ctxd.367.632955026")            = 0 <0.048564>
> unlink("ctxd.367.632955027")            = 0 <0.034952>
> unlink("ctxd.367.632955028")            = 0 <0.056535>
> unlink("ctxd.367.632955029")            = 0 <0.073922>
> unlink("ctxd.367.632955030")            = 0 <0.050084>
> unlink("ctxd.367.632955031")            = 0 <0.076422>
> unlink("ctxd.367.632955032")            = 0 <0.083707>
> unlink("ctxd.367.632955033")            = 0 <0.062325>
> unlink("ctxd.367.632955034")            = 0 <0.056119>
> unlink("ctxd.367.632955035")            = 0 <0.056179>
> unlink("ctxd.367.632955036")            = 0 <0.061726>
> unlink("ctxd.367.632955037")            = 0 <0.170615>
> unlink("ctxd.367.632955038")            = 0 <0.173498>
> unlink("ctxd.367.632955039")            = 0 <0.088021>
> unlink("ctxd.368.632955010")            = 0 <0.045662>
> unlink("ctxd.368.632955011")            = 0 <0.050733>
> unlink("ctxd.368.632955012")            = 0 <0.051693>
> unlink("ctxd.368.632955013")            = 0 <0.054854>
> unlink("ctxd.368.632955014")            = 0 <0.061138>
> unlink("ctxd.368.632955015")            = 0 <0.048361>
> unlink("ctxd.368.632955016")            = 0 <0.084508>
> unlink("ctxd.368.632955017")            = 0 <0.062291>
> unlink("ctxd.368.632955018")            = 0 <0.110628>
> unlink("ctxd.368.632955019")            = 0 <0.064505>
> unlink("ctxd.368.632955020")            = 0 <0.080547>
> unlink("ctxd.368.632955021")            = 0 <0.058048>
> unlink("ctxd.368.632955022")            = 0 <0.063159>
> unlink("ctxd.368.632955023")            = 0 <0.068933>
> unlink("ctxd.368.632955024")            = 0 <0.045993>
> unlink("ctxd.368.632955025")            = 0 <0.042045>
> unlink("ctxd.368.632955026")            = 0 <0.071292>
> unlink("ctxd.368.632955027")            = 0 <0.054626>
> unlink("ctxd.368.632955028")            = 0 <0.049143>
> unlink("ctxd.368.632955029")            = 0 <0.082537>
> unlink("ctxd.368.632955030")            = 0 <0.061396>
> unlink("ctxd.368.632955031")            = 0 <0.062972>
> unlink("ctxd.368.632955032")            = 0 <0.049971>
> unlink("ctxd.368.632955033")            = 0 <0.086166>
> unlink("ctxd.368.632955034")            = 0 <0.046925>
> unlink("ctxd.368.632955035")            = 0 <0.048068>
> unlink("ctxd.368.632955036")            = 0 <0.084247>
> unlink("ctxd.368.632955037")            = 0 <0.046254>
> unlink("ctxd.368.632955038")            = 0 <0.042681>
> unlink("ctxd.368.632955039")            = 0 <0.052280>
> unlink("d")                             = 0 <0.000059>
> unlink("e")                             = 0 <0.000050>
> unlink("f")                             = 0 <0.000048>
> unlink("g")                             = 0 <0.000049>
> unlink("h")                             = 0 <0.000047>
> unlink("i")                             = 0 <0.000046>
> unlink("j")                             = 0 <0.000046>
> unlink("k")                             = 0 <0.000050>
> unlink("l")                             = 0 <0.000045>
> unlink("m")                             = 0 <0.000041>
> unlink("n")                             = 0 <0.000035>
> unlink("o")                             = 0 <0.000033>
> unlink("p")                             = 0 <0.000036>
> unlink("q")                             = 0 <0.000035>
> unlink("r")                             = 0 <0.000034>
> unlink("s")                             = 0 <0.000034>
> unlink("t")                             = 0 <0.000035>
> unlink("u")                             = 0 <0.000036>
> unlink("v")                             = 0 <0.000035>
> unlink("w")                             = 0 <0.000035>
> unlink("x")                             = 0 <0.000035>
> unlink("y")                             = 0 <0.000034>
> unlink("z")
>
> -----Original Message-----
> From: Philip Rowlands [mailto:address@hidden
> Sent: Wednesday, September 26, 2007 8:43 AM
> To: Ken Naim
> Cc: address@hidden
> Subject: Re: Rm performance issue
>
> On Wed, 26 Sep 2007, Ken Naim wrote:
>
>> I am removing 300gb of data spread across 130 files within a single
>> directory and the process take just over 2 hours. In my past experiences
>> removing a small number of large files was very quick, almost
> instantaneous.
>> I am running red hat Linux on ibm p series hardware against a san with
> sata
>> and fiber drives. I see this issue on both the sata and fiber side
> although
>> the rm process is slightly faster on fiber.
>
> It's more likely to be caused by your storage system than rm, but here's
> how to tell:
>
> mkdir foo
> touch foo/{a,b,c,d,e,f}
> strace -T -e trace=file rm -rf foo
>
> Try this in a temporary directory on your local disk, to get some idea
> of how long the unlink(2) system call takes. Then try the strace on your
> slow-running rm command, and see how long rm is spending waiting for the
> system call to complete.
>
>
> Cheers,
> Phil




reply via email to

[Prev in Thread] Current Thread [Next in Thread]