Re: Memory leaks are observed for libgnutls in multi-thread mode

[Simon Josefsson]

(dropping libgcrypt-devel because this appears non-libgcrypt related
after all)

tangtong <address@hidden> writes:

> Hi,Nikos and Simon
>
> To verify the issue, I have configured my server to run as signle thread 
> mode. Under high TPS, the memory leak still happen in gnutls_handshake, which 
> means the root-cause is not caused by multi-thread. 
>
> By more logs and analysis, I have the following findings:
> Under high TPS, my server can't serve every session timely, which leads to 
> the closure of the sockets by the clients for timeout reason. The write 
> operation on the server side of the corresponding socket leads to broken pipe 
> error. gnutls_handshake() reports GNUTLS_E_PUSH_ERROR, -53. As a result, the 
> hand-shake stage of tls session is not finished successfully. 

Maybe that suggests a separate problem -- do you specify your own
push/pull functions?  Do they fail?

> After repeated testing, It is evident when aborted tls session caused by 
> error -53 are observed, the memory leak happen. 
>
> I have double check my codes, for these aborted session, I have called the 
> gnutls_bye()/gnutls_deinit() function.
>
> My assumption now is for those session which has unfinished hand-shake stage, 
> the resourses are not released properly in gnutls_handshake() for some reason.
>
> What's your opinion?

That seems plausible, we have only tried to fix memory leaks which we
have noticed.  Please provide a small standalone test code that
reproduce your problem, and it should be possible to fix it.  Look at
tests/mini.c, it may be useful as a starting point.

/Simon

> Regards
> Tony
>
>
>> Date: Tue, 13 Oct 2009 16:56:42 +0300
>> Subject: Re: Memory leaks are observed for libgnutls in multi-thread mode
>> From: address@hidden
>> To: address@hidden
>> CC: address@hidden; address@hidden
>> 
>> Hi,
>>  thanks for the investigation.
>> From the following trace:
>>                 libgcrypt.so.11`md_enable+0xfc
>>                 libgcrypt.so.11`md_open+0xfc
>> 
>> I suppose that this leak occurs in libgcrypt md_enable() in md.c. I
>> cannot figure out where exactly though. I CC the gcrypt-devel list for
>> more insight.
>> 
>> best regards,
>> Nikos
>> 
>> 2009/10/12 tangtong <address@hidden>:
>> > I have redone my test and go through the memory leak points, I get the
>> > following info:
>> >> ::findleaks
>> > CACHE     LEAKED   BUFCTL CALLER
>> > 00204e08       1 004ab7e8 libclntsh.so.10.1`sigpnm+0x80
>> > 0020b188    7816 007f53b0 libgcrypt.so.11`do_malloc+0x54
>> > 0020ae08     106 0130e960 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08       1 00c0cd98 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08      63 008a5e78 libgcrypt.so.11`do_malloc+0x54
>> > 0020ae08    8153 0043f518 libgcrypt.so.11`do_malloc+0x54
>> > 0020b188     422 01046168 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08    8330 00b3d860 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08    8230 01206438 libgcrypt.so.11`do_malloc+0x54
>> > ----------------------------------------------------------------------
>> >    To! tal   33122 buffers, 21130336 bytes
>> >
>> >> 007f53b0$<bufctl_audit
>> > 0x7f53b0:       next            addr            slab
>> >                 0               7f3700          21aa50
>> > 0x7f53bc:       cache           timestamp       thread
>> >                 20b188          738886035200566511        &nb! sp;
>> > 0x7f53cc:       lastlog         contents        stackdepth
>> >                 1d8000          0               15
>> >                 libumem.so.1`umem_cache_alloc+0x208
>> >                 libumem.so.1`umem_alloc+0x44
>> >                 libumem.so.1`malloc+0x2c
>> >                 libgcrypt.so.11`do_malloc+0x54
>> >       &nbs! p;         libgcrypt.so.11`_gcry_malloc+0x10
>> >                 libgcrypt.so.11`md_enable+0xfc
>> >                 libgcrypt.so.11`md_open+0xfc
>> >                 libgcrypt.so.11`_gcry_md_open+0x4c
>> >                 libgnutls.so.26`wrap_gcry_hash_init+0x60
>> >                 libgnutls.so.26`_gnutls_hash_init+0x78
>> >                 libgnutls.so.26`gnutls_handshake+0xe8
>> >                 libUE.so`_ZN12SSLSETDriver9onRec! eiveEv+0x268
>> >                 libUE.so`_ZN12InTaskRunner3runEv+0x118
>> >                 libclassutil.so`_ZN7MThread7routineEv+0x28
>> >                 libclassutil.so`_ZN7MThread10thrRoutineEPv+0x1c
>> >
>> > All other leaks points also show the same clues: memory leaks happen during
>> > the gnutls_handshake.
>> >
>> > For the report of MDB, total 21130336 bytes memory leaks are observed. I
>> > have launched 167201 session in 3344 seconds.
>> >
>> > Anybody can give me some helps? If I am not using gnutls in the proper
>> > way???
>> >
>> > Regards
>> > Tony
>> >
>> > ________________________________
>> > From: address@hidden
>> > To: address@hidden
>> > Date: Sat, 10 Oct 2009 08:21:05 +0000
>> > Subject: Memory leaks are observed for libgcrypt.so.11 in multi-thread mode
>> >
>> > Hi,
>> > My program is a multi-thread server(pthread) working in Solaris enviorment,
>> > For thread-safe consideration, according to the guide, I have defined the
>> > following macro and call the specific function during iniatlization:
>> > GCRY_THREAD_OPTION_PTHREAD_IMPL;
>> >  gcry_control (GCRYCTL_SET_THREAD_CBS, &gcry_threads_pthread);
>> >
>> > Scenario1:
>> > Launch Tls session one after another to guarantee there is no concurrency
>> > existing between tls session, there is no memory leak reported by MDB;
>> >
>> >
>> > Scenario2:
>> > Launch TLS session concurrently, e.g., 50 TPS,  memory leaks are reported 
>> > by
>> > MDB
>> >
>> >> ::findleaks
>> > CACHE     LEAKED   BUFCTL CALLER
>> > 00204a88      17 0053b860 libUE.so`_ZN12PacketHelper12createPacketEi+0x34
>> > 0020dc08      27 00aea708 libgcrypt.so.11`do_malloc+0x54
>> > 0020b188      88 012f0b40 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08&n! bsp;    100 013aa000 libgcrypt.so.11`do_malloc+0x54
>> > 0020ae08      64 00461e00 libgcrypt.so.11`do_malloc+0x54
>> > 0020b188      39 0073a780 libgcrypt.so.11`do_malloc+0x54
>> > 0020ae08      65 016cf248 libgcrypt.so.11`do_malloc+0x54
>> > 0020dc08     129 00aea7f8 libgcrypt.so.11`do_malloc+0x54
>> > ----------------------------------------------------------------------
>> >    Total     529 buffers, 325752 bytes
>> >
>> > I have disabled the session reusage and deinit tls sessions structure with
>> > gnutls_deinit().
>> >
>> > Anybody can give me some tips on this issue?
>> >
>> > Regards
>> > Tony
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > ________________________________
>> > 使用新一代 Windows Live Messenger 轻松交流和共享！ 立刻下载！
>> > ________________________________
>> > 使用新一代 Windows Live Messenger 轻松交流和共享！ 立刻下载！
>> > _______________________________________________
>> > Help-gnutls mailing list
>> > address@hidden
>> > http://lists.gnu.org/mailman/listinfo/help-gnutls
>> >
>> >
>                                         
> _________________________________________________________________
> 上Windows Live 中国首页，下载最新版Messenger！
> http://www.windowslive.cn_______________________________________________
> Help-gnutls mailing list
> address@hidden
> http://lists.gnu.org/mailman/listinfo/help-gnutls