Re: [Discuss-gnuradio] Recurring memory leak problems with iterative decoding [ GNU Radio 3.7.11.1]

[Ben Hilburn]

Hey Moses -

I don't have an immediate answer for you, but it seems likely the issue is with `decoded_u8`. Before spending time trying to debug why this might be happening when the code is used in GNU Radio versus not, it would be good to figure out where exactly the leak is occurring.

You should be able to test the `decoded_u8` hypothesis by commenting out the existing malloc & free, and just using some hard-coded dummy vector or something similar. Your application obviously won't work, but what you care about is how that affects the memory leak.

Separately, is there a reason you are dynamically allocating that vector? You are freeing the memory within the same scope, anyway. I guess I'm not sure how much data that realistically is, so perhaps that's why you're putting it on the heap?

Cheers,

Ben

On Wed, May 8, 2019 at 1:33 PM Moses Browne Mwakyanjala <address@hidden> wrote:

Hello Ben,

Yes. There are no memory leaks when the block is disabled. 

Regards,

Moses. 

On Wed, May 8, 2019 at 5:50 PM Ben Hilburn <address@hidden> wrote:

Hi Moses -

And just to confirm, if you remove your LDPC block from that flowgraph or replace it with a passthrough, you don't see the leak?

Cheers,

Ben

On Tue, May 7, 2019 at 7:24 PM Moses Browne Mwakyanjala <address@hidden> wrote:

Hello Ben,

Thanks.

For LDPC, the executable can be found at

gr-ccsds/examples/LDPC/ldpc_2/build-ldpc_decoder-Desktop-Debug/ldpc_decoder.

The C++ executable for Turbo code can be found at 

gr-ccsds/lib/fec/turbo/deepspace-turbo/bin/deepspace_turbo

I'm not very familiar with Valgrind so I monitored the memory usage by looking at system monitor on my Ubuntu laptop. The memory usage is almost constant, at around 17.1 Mbs for the ldpc_decoder executable. On GNU Radio, the memory usage jumps by huge steps (100Mb) in a matter of seconds until all the memory (the ram is around 8 gigs) is fully consumed. 

Thanks for links to the memory buffer blog post. I will have a look. 

Regards,

Moses. 

On Tue, May 7, 2019 at 10:13 PM Ben Hilburn <address@hidden> wrote:

Hey Moses -

This is really cool work! Thanks so much for sharing it. Michael's suggestion of pushing it was a good one. I haven't looked at the code yet, but:

The code was able to run smoothly in a C++ application but experienced memory leaks in GNU Radio.

I'm curious how confident you are in this? It might be worthwhile to run the pure-C++ version through Valgrind just to double-check, if you haven't already.

I also have one question regarding buffering in GNU Radio. Since iterative decoding with a large number of iterations and large block sizes takes time to complete, the input pmt data that is not consumed immediately will have to be stored somewhere. Is that the case? Could that be the reason for the memory leak?

Things do get stored until buffers and full, and then backpressure builds up through the flowgraph. This shouldn't cause memory leaks.

For a more thorough explanation of this, check out this excellent blog post from Marcus Mueller!

https://www.gnuradio.org/blog/2017-01-05-buffers/

Cheers,

Ben