Re: [lwip-users] memory pools question

[Barry Andrews]

I must say that I agree with one thing here, the description of how the
pools and heap should be configured are at the least vague.
So many answer in this group say it depends upon your usage but in no
way explain how.

There really needs to be a document which explains what takes from the
heap and what takes from the pools and the effect of too small a number.

For instance most pools it seems like effect is basically just a dropped
packet, so non fatal but heap seems fatal all the time.

It would be nice to have one place where this is clearly documented so
you can make an informed decision as to pool sizes rather than well it
works so that must be fine.

Regards
Barry Andrews

-----Original Message-----
From: address@hidden
[mailto:address@hidden On
Behalf Of Artem Pisarenko
Sent: Friday, 16 November 2012 20:12
To: address@hidden
Subject: Re: [lwip-users] memory pools question

Hello.

I need help again.
After I configured lwipopts.h in right way(I thought I did), it turned
out that lwip need LWIP_MALLOC_MEMPOOLs.
Furthermore, their sizes are unknown for me. I was able to count every
regular pool (type-defined) but these ones somehow required by TCP
windowing and retransmissions mechanisms. I have no idea what sizes they
should be ?

Generally speaking, pools always imply that their sizes must be
calculable by developer to make sure they will match memory usage
pattern of application and fulfill memory constraints. Lwip pools are
completely mysterious for developer not delved into the innards of lwip.

Even for me, after I spent few months having analysis of lwip code from
time to time. This makes pools unusable despite the fact they already
implemented. I suggest most developers (including me) use random values
at this moment. Pools purpose is that they should protect application
from memory fragmentation which leads to failure. But lwip pools
configured to random values may cause "out of memory" error which leads
to absolutelly same failure.

Please, somebody throw light upon this subject once more :)

Regards, Artem.


25.01.2012 13:52, Simon Goldschmidt wrote:
> jblackarty <address@hidden> wrote:
>> It's totally unclear are mem and memp independent mechanisms, or mem 
>> utilizes memp, or memp uses mem, or they are cross-references each 
>> other ? Custom pools are part of pools (memp), or pools are part of 
>> custom pools ?
>>
>> There are complicated set of options:
> Yes, that's the nature of lwIP being adjustable to the needs of your
(possibly small or otherwise restricted) target hardware.
>
> I'll start explaining the 'standard' options. Basically, lwIP has 2
strategies for memory allocation: pools and a heap. The heap is mainly
only used for outgoing PBUFs allocated with the type PBUF_RAM. Then
there is one memory pool per structure type:
>
>> MEMP_NUM_*,
> These control the amount of structures of that type (e.g.
MEMP_NUM_UDP_PCB sets the amount of available udp connections).
>
>> MEM_SIZE,
> This is the size of the heap (used for PBUF_RAM allocation).
>
>> PBUF_POOL_SIZE.
> As it says in opt.h (did you read that file???), the number of pbufs
in the pbuf pool (where PBUF_POOL_BUFSIZE controls the size of these
pbufs). The pbufs from the pool are of type PBUF_POOL.
>
>
> Nnow to the more advanced options:
>
>> MEM_LIBC_MALLOC,
> Some people don't care about the danger of heap fragmentation and want
to use the C library's malloc/free functions instead of the
lwIP-internal memory allocation (if only to save code because the C
library's functions are included in their binary, anyway). Set this to 1
to do so.
>
>> MEMP_MEM_MALLOC,
> This is mainly only usefule with the previous option: when set to 1,
memp pool allocation uses the heap allocation functions (this was
introduced to let memp allocations be using the C library's malloc.
>
>> MEM_USE_POOLS,
> This is the other way round than the previous: when you're scared of
heap fragmentation, let mem_malloc() allocate from specific pools. The
size of these pools (and the number of entries in them) needs to be
configured *and* you have to enable the next option.
>
>> MEMP_USE_CUSTOM_POOLS,
> This enables custom memp pools. They are needed for the previous
option but you can also define more pools, for example to put your
semaphores or mboxes into.
>
>> How they related to each other ?
>> For MEMP_MEM_MALLOC a comment says "Use mem_malloc/mem_free instead 
>> of the lwip pool allocator.". But mem_malloc is part of pool
mechanisms.
> No, memp_malloc (not the 'p') allocates from pools, mem_malloc
allocates from the heap!
>
>> How it can be disabled ? Also "Especially useful with MEM_LIBC_MALLOC

>> ..." confuses. Isn't libc allocator completely disables pool 
>> mechanism ? These options must exclude each other.
> No, why should they???
>
>> Is it possible disable allocating standard structs in pool at all ?
> Yes, see above
>
>> Does it makes sense configure MEMP_NUM_* options if MEM_LIBC_MALLOC 
>> enabled ?
> No.
>
>> MEM_SIZE includes only pool of standart structures or also pbufs pool
?
> None of them. It controls the heap size and has nothing todo with
pools.
>
>> Questions may be continued indefinitely.
> Is that so?
>
>> lwip source code doesn't
>> describe architecture because it's heavily stuffed with complex macro

>> conditions nested each in other.
> lwIP relies on people's contribution, so if there's anything missing
for a beginner, please don't hesitate to contribute. Writing something
on the wiki is the easiest part to help future beginners getting started
faster.
>
>> "Custom_memory_pools" article on wiki doesn't
>> help too because it doesn't describe whole memory mechanism
completely.
> Right, it only describes how to enable custom memory pools. I guess
it's rather targeted at people already using lwIP as a guideline of how
to get custom pools running.
>
>> I will appreciate if someone throw light upon this subject.
> Hope I did. :-)
>
> Simon


_______________________________________________
lwip-users mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/lwip-users