[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Monotone-devel] memory leak in Botan::Pipe?
From: |
Nathaniel Smith |
Subject: |
[Monotone-devel] memory leak in Botan::Pipe? |
Date: |
Sat, 2 Sep 2006 08:26:32 -0700 |
User-agent: |
Mutt/1.5.12-2006-07-14 |
We were noticing some serious resource usage problems in 'mtn
checkout' on large trees, and the evidence seems to be pointing to
some sort of problem in Botan. On a checkout of h:net.venge.monotone,
a medium-sized tree, valgrind reports ~5 megabytes of leaked data, all
of data allocated at line 189 of Botan's pipe.cpp:
void Pipe::find_endpoints(Filter* f)
{
for(u32bit j = 0; j != f->total_ports(); ++j)
if(f->next[j] && !dynamic_cast<SecureQueue*>(f->next[j]))
find_endpoints(f->next[j]);
else
{
SecureQueue* q = new SecureQueue; // <---- right here
f->next[j] = q;
outputs->add(q);
}
}
The application code that (eventually) calls find_endpoints and
triggers the leaks is (with comments and asserts removed):
{
ofstream file(tmp.as_external().c_str(),
ios_base::out | ios_base::trunc | ios_base::binary);
Botan::Pipe pipe(new Botan::DataSink_Stream(file));
pipe.process_msg(dat());
}
This doesn't _look_ like we're doing anything wrong to me... I don't
understand the memory handling strategy that Pipe is using, though, so
I don't know where the actual bug is.
This problem was initially encountered when doing tests on a giant
repository (100,000+ files, 1+ gigabytes), where monotone mysterously
ended with hundreds of megabytes of resident memory, presumably
because of this leak; so this is a relatively serious problem.
(This massive leak also triggered pathological behavior in Botan's
memory management routines -- somewhere over 97% of cpu time was
being spent in Pooling_Allocator::deallocate. Glibc's default
heap routines were able to scale to this load easily, though, when I
tried swapping in a trivial malloc-based allocator -- is there really
any reason to use Pooling_Allocator for unlocked memory?)
-- Nathaniel
--
"But in Middle-earth, the distinct accusative case disappeared from
the speech of the Noldor (such things happen when you are busy
fighting Orcs, Balrogs, and Dragons)."
- [Monotone-devel] memory leak in Botan::Pipe?,
Nathaniel Smith <=