Re: [Discuss-gnuradio] gr::buffer::allocate_buffer: warning

[Anderson, Douglas J.]

Marcus,

Awesome! I'm working my way through the scheduler slides linked earlier today, and I get to where it's describing buffers on linux like "attach shmid1 to first half of shmid2 with shmat, attach shmid1 to second half of shmid2 with shmat, memory in both halves of shmid2 are mapped to the same virtual space..." and I was just really lost. I appreciate the diagram; now it's making sense that the circular buffer is implemented with 2 copies of "shmid1" back to back. I see now that the page size if just a requirement of the tools used to create this clever buffer:

shmget() returns the identifier of the shared memory segment associated with the value of the argument key. A new shared memory segment, with size equal to the value of size rounded up to a multiple of PAGE_SIZE

Very cool stuff. Thanks!

-Doug

---

From: discuss-gnuradio-bounces+address@hidden [discuss-gnuradio-bounces+address@hidden on behalf of Marcus Müller address@hidden
Sent: Tuesday, April 21, 2015 12:33 PM
To: address@hidden
Subject: Re: [Discuss-gnuradio] gr::buffer::allocate_buffer: warning

Hi Doug,

ok, you asked for this :D
So, GNU Radio's buffers look a lot like real circular buffers to the blocks using them:
For example, assume your buffer between source block A and sink block B is large enough to store exactly 10000 of your items:
A-->B
Now, A has produced 9000 items, of which B has already consumed 8000. Your buffer thus looks like this:

- - - - - - - - R W -

With R being the current position of the read pointer, i.e. the address of the first input item of the next B::work() call,
and W being the write pointer, i.e. the address of the first output_item on the next A::work() call.
Each "- " (or "R " or "W ") is 1000 items worth of storage.

Now, we can agree that in this buffer, there are thousand items that must not be overwritten by the next A::work call, and 9000 items space for new items.

What GNU Radio does is employ memory mapping magic to allow A to simply write contigously the next 9000 items; to the process, the memory looks like this:

- - - - - - - - R W -|- - - - - - - - R W -

Notice that the second half is really just a transparent image of the first, being inserted there by the memory control unit of your CPU.

Now, that is awesome on so many levels:
* it allows developers to always write their applications like the in- and output was just linear in memory, no matter where in their buffer they are
* it allows full usage of the buffer, without having extra space allocated

Of course, this has an architectural downside:
On any fully-fledged general purpose CPU platform I know, you can only do this with pages; a page is simply the smallest amount of memory you can tell your memory management unit to map somewhere.
On Linux, these pages are generally 4KB. That's usually a bit handy, because it's a power of two bytes, which means that the typical item sizes (1B for a byte/char, 2B for a short, 4B for a float or int, and 8B for a std::complex<float> == gr_complex) fit well here, but not so cool if your item's size is not a divider of 2**12. In that case, the scheduler has no chance but to find the smallest common multiple of 4096 and your item size.

Greetings,
Marcus

On 04/21/2015 08:04 PM, Anderson, Douglas J. wrote:

Would it be possible to dive into this a bit deeper? I'm trying to get more familiar with the the scheduler and how the buffer is laid out.

So using "tried to allocate 41 items of size 1592. Due to alignment requirements 512 were allocated" as an example, the scheduler looked at the requirements of the various blocks (or does each block have a dedicated buffer?) and decided it needed buffers of at least 41x1592 bytes? But assuming no block explicitly called set_alignment(), the scheduler uses the default page size (4096 bytes) and just allocated a number of 1592-byte blocks such that (n * 1592) % 4096 == 0?

Am I on the right track with this?

-Doug

---

From: discuss-gnuradio-bounces+address@hidden [discuss-gnuradio-bounces+address@hidden] on behalf of Ali Riaz [address@hidden]
Sent: Monday, April 20, 2015 12:16 PM
To: address@hidden
Subject: [Discuss-gnuradio] gr::buffer::allocate_buffer: warning

Hello everyone,

I'm getting the following warnings when running my application, does anyone know what this means?

gr::buffer::allocate_buffer: warning: tried to allocate
   41 items of size 1592. Due to alignment requirements
   512 were allocated.  If this isn't OK, consider padding
   your structure to a power-of-two bytes.
   On this platform, our allocation granularity is 4096 bytes.
gr::buffer::allocate_buffer: warning: tried to allocate
   82 items of size 796. Due to alignment requirements
   1024 were allocated.  If this isn't OK, consider padding
   your structure to a power-of-two bytes.
   On this platform, our allocation granularity is 4096 bytes.
gr::buffer::allocate_buffer: warning: tried to allocate
   82 items of size 796. Due to alignment requirements
   1024 were allocated.  If this isn't OK, consider padding
   your structure to a power-of-two bytes.
   On this platform, our allocation granularity is 4096 bytes.
gr::buffer::allocate_buffer: warning: tried to allocate
   82 items of size 796. Due to alignment requirements
   1024 were allocated.  If this isn't OK, consider padding
   your structure to a power-of-two bytes.
   On this platform, our allocation granularity is 4096 bytes

Best,

Ali

_______________________________________________
Discuss-gnuradio mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/discuss-gnuradio