Re: [Cvs-dev] [PATCH, one-liner] memory leak bug (buffer.c:buf_free_data) in all CVS versions

[Sergei Trofimovich]

On Mon, 14 Jan 2013 18:09:13 +1100
"Arthur Barrett" <address@hidden> wrote:

> Sergei,
> 
> Thanks for the report.
> 
> But there are some problems.
> 
> 1. you didn't attach a patch.
My bad. Reattached.
Backup link if I have broken something again:
    
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-vcs/cvs/files/cvs-1.12.12-fix-massive-leak.patch?revision=1.1

> 2. CVS 1.11.23 buffer.c does not have buf_free_data
Seems to be introduced around 2004:

> Author: dprice <dprice>
> Date:   Thu Sep 2 16:24:56 2004 +0000
> 
>     * server.c (do_cvs_command): Pass new args to fd_buffer_initialize().
> ...

> 3. CVS 1.12 buffer.c does not have buf_free_data
The version I discovered was a cvs-1.12.12 (newer ones
up to CVS cvs tree still contain the bug as far as I see.
Looking at rsync://cvs.savannah.gnu.org/sources/cvs/)

> 4. CVS 2.x/CVSNT doesn't have buf_free_data
It's not about this software, right?
    http://ftp.gnu.org/non-gnu/cvs/source/

> 5. cvsps is nothing to do with CVS (though the functionality is already
> in CVSNT for 5 years or more)

cvsps is a tool to export cvs repo as atomic changesets eatable
by 'git fast-import' for example.
It connects to CVS server (or runs it locally for :local: CVSROOT)
and runs roughly the following commands:

    <connect to host>
    cvs rlog proj # parse revisions
    cvs co -r 1.1 file
    cvs co -r 1.2 file
    <disconnect from host>

> If you can submit the missing information (patch to CVS 1.12.13 or CVS
> 1.11.23) that may help.

> 
> Regards,
> 
> 
> Arthur Barrett
> Product Manager, CVSNT
> 
> 
> > -----Original Message-----
> > From: 
> > address@hidden 
> > [mailto:address@hidden
> > rg] On Behalf Of Sergei Trofimovich
> > Sent: 12 January 2013 08:47
> > To: address@hidden
> > Subject: [Cvs-dev] [PATCH, one-liner] memory leak bug 
> > (buffer.c:buf_free_data) in all CVS versions
> > 
> > 
> > Hello ladies and gentlemen!
> > 
> > In my recent exploration on tools
> > reading entire cvs history I've
> > noticed horrible leak.
> > 
> > It allows to suck out all RAM out
> > of your cvs server just by issuing
> > any (say, cvs checkout) simple command
> > many times.
> > 
> > A description of an attached patch with
> > some numbers:
> > 
> >     buf_free_data must free data independently
> >     of send or reseived bytes over network.
> >     
> >     Moreover, when buffer is usually freed
> >     buffer _is_ empty, but has one clean mapped page.
> > 
> >     I've observed massive 'cvs server' leaks
> >     when importing large gentoo-x86 repo with 'cvsps'.
> >     Leak ate all my 32GBs of RAM and killed process.
> >     (Leaked around 3 pages per client request).
> >     
> >     valgrind found the leak easily:
> >     
> >     $ valgrind \
> >         cvsps \
> >                --root :local:$HOME/portage/gentoo-x86.rsync \
> >                --fast-export \
> >                gentoo-x86/dev-vcs/git-annex 2>l |
> >         git fast-import
> > 
> >         ==13504== 1,248 bytes in 52 blocks are still 
> > reachable in loss record 41 of 47
> >         ==13504==    at 0x4C2C19B: malloc (vg_replace_malloc.c:270)
> >         ==13504==    by 0x48A556: xnmalloc_inline (xmalloc.c:40)
> >         ==13504==    by 0x48A5B5: xmalloc (xmalloc.c:56)
> >         ==13504==    by 0x4855F5: new_memnode (pagealign_alloc.c:91)
> >         ==13504==    by 0x48571B: pagealign_alloc 
> > (pagealign_alloc.c:151)
> >         ==13504==    by 0x485739: pagealign_xalloc 
> > (pagealign_alloc.c:182)
> >         ==13504==    by 0x408DD7: get_buffer_data (buffer.c:98)
> >         ==13504==    by 0x409C0C: buf_input_data (buffer.c:738)
> >         ==13504==    by 0x45BB63: do_cvs_command (server.c:3847)
> >         ==13504==    by 0x45D39E: serve_co (server.c:4809)
> >         ==13504==    by 0x45F845: server (server.c:6438)
> >         ==13504==    by 0x438784: main (main.c:1066)
> > 
> >     And now it takes constant space (less, than 18MB)
> >     for 'cvs server' process to convert all gentoo-x86
> >     by serving more, than 5 000 000 client requests.
> > 
> >     Signed-off-by: Sergei Trofimovich <address@hidden>
> > 
> > Thanks!
> > 
> > -- 
> > 
> >   Sergei
> > 
> 
> _______________________________________________
> Cvs-dev mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/cvs-dev
> 

-- 

  Sergei

cvs-1.12.12-fix-massive-leak.patch
Description: Text Data

signature.asc
Description: PGP signature