[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [bug #23023] cp -r use a lot of memory, feature or memory leak?
From: |
Jim Meyering |
Subject: |
Re: [bug #23023] cp -r use a lot of memory, feature or memory leak? |
Date: |
Thu, 01 May 2008 15:21:49 +0200 |
Matteo Boccafoli <address@hidden> wrote:
> URL:
> <http://savannah.gnu.org/bugs/?23023>
> Summary: cp -r use a lot of memory, feature or memory leak?
...
> # 21000 dir in 1.tar.bz2
> $ cd /dev/shm
> /dev/shm$ time tar -xjf 1.tar.bz2
>
> real 0m8.403s
> user 0m3.188s
> sys 0m5.196s
>
> /dev/shm$ time ./busybox cp -r /dev/shm/1/ /dev/shm/b
> real 0m54.421s
> user 0m0.832s
> sys 0m51.751s
> # [time t_1,t_2 : t_1<t_2 and cp.memory(t_1)==cp.memory(t_2)]
>
> /dev/shm$ time /bin/cp -r /dev/shm/1/ /dev/shm/c
> real 0m55.696s
> user 0m2.680s
> sys 0m50.447s
> #[time t_1,t_2 : t_1<t_2 and cp.memory(t_1)<cp.memory(t_2)]
> # using a lot of memory (max 54 MB of RAM)
>
> Is it a feature or memory leak?
As far as I know, cp has no memory leaks.
However, a peak memory usage of 54MB does sound high.
How did you measure it?
I know that "busybox cp" fails to preserve some hard-link
relationships that GNU cp does preserve. That may explain
some of the difference.
Note however, that cp is far from optimal.
There is work underway to rewrite is using fts.c,
and that should reduce its memory consumption in addition
to making it more robust.
I tried to duplicate your situation as follows but ended up
with a peak memory usage of just 3.3 MB.
(BTW, how many *non-directories* are in your tarball?
Count with this command: find 1 -type f|wc -l )
# Create a hierarchy of 21000 directories, each containing
# three empty files named a, b, and c.
$ ( rm -rf /tmp/1; mkdir /tmp/1 && cd /tmp/1 && seq 21000 \
|xargs mkdir && seq 21000|xargs -i% echo %/a %/b %/c|xargs touch )
# run cp under valgrind's memory profiler
$ rm -rf massif.* /tmp/2; valgrind --tool=massif ./cp -r /tmp/1 /tmp/2
# display the first few lines of results (full output is over 2200 lines)
$ ms_print massif.out.*
--------------------------------------------------------------------------------
Command: ./cp -r /tmp/1 /tmp/2
Massif arguments: (none)
ms_print arguments: massif.out.30264
--------------------------------------------------------------------------------
MB
3.273^ ,#
| ,@@@@#
| .,@@@@@@@#
| .. ::@@@@@@@@#
| .@:: ::@@@@@@@@#
| .. : :@:: ::@@@@@@@@#
| .: :: : :@:: ::@@@@@@@@#.
| , :: :: : :@:: ::@@@@@@@@#:
| ...@:@ :: :: : :@:: ::@@@@@@@@#:
| ..: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| ..::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| @ .@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| , . : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| @: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| . ,@@: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| ,. : @@@: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| .. :@: : @@@: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| ,.@:: :@: : @@@: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
| .,@ @:@:: :@: : @@@: :: : @:: :@ ::::: :::@:@ :: :: : :@:: ::@@@@@@@@#:
0 +----------------------------------------------------------------------->Mi
0 155.8
Number of snapshots: 63
Detailed snapshots: [2, 3, 4, 6, 10, 14, 15, 16, 21, 25, 34, 36, 43, 48, 50,
51, 52, 53, 54, 55, 56, 57, 58, 59, 60 (peak)]
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: [bug #23023] cp -r use a lot of memory, feature or memory leak?,
Jim Meyering <=