[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#16309: cp is a cpu hog

From: Pádraig Brady
Subject: bug#16309: cp is a cpu hog
Date: Tue, 31 Dec 2013 21:24:13 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130110 Thunderbird/17.0.2

On 12/31/2013 07:37 AM, U.Mutlu wrote:
> Hello,
> there is a big bug with the cp command: it eats all cpu resources,
> the system is not responsive while copying a huge file,
> say a file of size 10 GB.
> There are other _desasterous_ consequences of this behaviour,
> if interessted I can tell you.
> OS: Debian, 64bit, latest, happens also on other distros like Ubuntu,
> see for example these links:
>  http://ubuntuforums.org/showthread.php?t=1206936
>  https://bugs.launchpad.net/ubuntu/+source/linux/+bug/365775
> http://stackoverflow.com/questions/4290679/why-high-io-rate-operations-slow-everything-on-linux
>  ...
> I suggest you just place a nanosleep(a_small_value)
> in the loop that does the copy operation.
> From my experience in simular situations this helps much.

In my experience if a process consumes all available CPU,
it has little negative impact on system responsiveness.
If however I/O resources are saturated, then the system
can become unresponsive.  So I'd first like to see confirmation
of cp consuming all CPU (which isn't necessarily a bad thing anyway).

Also I've noticed a couple of related issues in the Linux VM.
1. It doesn't handle large files very well at present, caching very large
portions (given today's common large RAM sizes) before starting to write out.
2. The resultant writes block other system (I/O) operations for long periods of 

You could avoid the Linux VM by using direct I/O.
For example you could see if you had the same issue with dd:

  dd bs=1M if=big.file of=/the/copy

Then see if direct I/O improves things:

  dd bs=1M if=big.file of=/the/copy oflag=direct

You could also check if streaming without polluting the cache helps
(while taking advantage of the readahead cache) using:

  dd bs=1M if=big.file of=/the/copy iflag=nocache oflag=nocache


reply via email to

[Prev in Thread] Current Thread [Next in Thread]