Re: [lwip-users] recv function BLOCK because recv_mbox is full: can yougive me a solution?

[Bikram Chatterjee]

Perro,

I dug up some details of my problem. It looks like matched your symptom.

mbox (not recvmbox) is used for different purposes,
1. invoking do_* functons from stack
2. sending messages from driver to stack (netif->input())

What happened in my case is:
1. I had an HAL which run in a loop in a thread and on recv event from
driver, gets the packet from driver and calls netif->input() to pass
the pbuf to tcpip_thread for processing in stack.
2. In tcpip_thread the pbuf is taken from mbox and after finding
suitable conn it is put into its conn->recvmobx
at the same time...
1a. application calls recv which posts a message with do_recv callback to mbox.
2a. tcpip_thread fetches this callback and invokes it -- this call
back eventually fetches from conn->recvmbox and copies the pbuf into
application supplied buffer.

Now if mbox is small and driver is producing packet faster than
application is consuming them for each packet mbox contains two
message.

Consider in your case, if your driver produces 9 packets (8 is the max
size of all mailboxes) in time, in which application can consume only
one packet. The following scenario occurs:
1. After consuming one buffer-full application goes for the second time.
2. By that time driver have already posted 8 messages in mbox and
blocking for 9th packet.
3. Application can not post anymore callbacks in mbox as it is already
full and waiting on mbox.
4. tcpip_thread has already posted 8 messages into recvmbox and has
9th message in hand from driver to post in recvmbox and blocked.

[NB: Counting maybe not not correct, but you got the idea]

I have used some flow control in driver, together with variable sized
mailboxes (bigger mailbox for mbox only) to optimize the throughput
and avoiding the deadlock.

A better solution would be to ensure deadlock never happens. For
example from driver never post more than n - 1 messages into mbox,
where n is recvmbox size. mbox has a size greater than 2n etc.

I hope these helps.

Bikram



On Fri, Mar 7, 2008 at 10:31 PM, Piero 74 <address@hidden> wrote:
> Bikram,
>
> after changing of recvmbox size ( 8 ---> 20 !!! ) the problem seems
> disapper....
> i tried to increase again a throughput, and i saw the problem agian.... but
> i had a breakpoint in my debugger in sys_arc_trypost error and debugger
> doesn't stop.
>  After your post, i check lwip code:
>
> as you said... it's possible that mbox had problem... because there are some
> calls: sys_mbox_post(mbox, &msg);
> this calls block forever....
>
> i want to suggest to LWIP developer:
>
> change ALL calls to sys_mbox_post in sys_mbox_trypost
> if sys_mbox_trypost fails for a lot of time, caller drops all msgs in
> xxxx_mbox
>
> if you can, forward thi idea to lwip developer...
>
> i will continue to investigate
>
> bye,
> Piero
>
> 2008/3/7, Bikram Chatterjee <address@hidden>:
>
> > I am not sure but some times back I ran into a similar problem. I
> > remember to trace it back to a deadlock due to small size of mbox.
> >
> > I doubt that it not recvmbox but it is mbox that is getting filled up
> > and posting new commands from application and driver are blocked up.
> >
> > But that might not be the case here, just though it might help, at to
> > be eliminated as a possibility.
> >
> >
> > On Fri, Mar 7, 2008 at 9:45 PM, Piero 74 <address@hidden> wrote:
> > >
> > >
> > >
> > > 2008/3/7, Bill Auerbach <address@hidden>:
> > >
> > > >
> > > >
> > > >
> > > >
> > > > Should receive have blocked at all since there is data in the
> recv_mbox?  Overflowing has no bearing on a thread waiting for data when
> there is data already in the mbox.
> > >
> > >
> > > yes... it has blocked.... i tried to  use  a recv_mbox with more
> elemets, and the problem dosn't  happen... but  i'm thinking something to
> avoid this block...
> > >
> > > i don't know if can help you to understand the problem, but i saw:
> > >
> > > tcp fast timer was running and calling sys_mbox_trypost, but mbox was
> full... so tcp_ip thread was running, but application task was blocked... in
> think in sys_arch_mbox_fetch (called with timeout =0) in netconn_recv.
> > >
> > > Do you think could be usefull this:
> > >
> > > enable LWIP_SO_RCVTIMEO
> > > set SO_RCVTIMEO option for a socket
> > >
> > > i think (i hope) in this way sys_arch_mbox_fetch will never call with
> timeout =0.... what do you think?
> > >
> > > bye
> > > Piero
> > >
> > >
> > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Bill
> > > >
> > > >
> > > >
> > > >
> > > > ________________________________
> >
> > > >
> > > > From: address@hidden
> [mailto:address@hidden On Behalf
> Of Piero 74
> > > > Sent: Friday, March 07, 2008 10:02 AM
> > > > To: Mailing list for lwIP users
> > > > Subject: [lwip-users] recv function BLOCK because recv_mbox is full:
> can yougive me a solution?
> > > >
> > > >
> > > >
> > > >
> > > > Hi all
> > > >
> > > > this is my new problem:
> > > >
> > > > I have a task which calls lwip_recv with the flag MSG_DONTWAIT.
> > > >
> > > > I saw a strange situation: my recv_mbox is full, so sys_mbox_trypost
> return ERR_MEM, so recv_tcp return ERR_MEM and tcp packet is dropped... but
> recv remains BLOCKED! In this situation my code cannot check the problem and
> close the socket!
> > > >
> > > > I can set size for recv_mbox using high value, but it's not a
> solution.... can anyone explain how i can trap this problem?
> > > >
> > > > thanks,
> > > > Piero
> > > > _______________________________________________
> > > > lwip-users mailing list
> > > > address@hidden
> > > > http://lists.nongnu.org/mailman/listinfo/lwip-users
> > > >
> > >
> > >
> > > _______________________________________________
> > > lwip-users mailing list
> > > address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/lwip-users
> > >
> >
> >
> >
> >
> > --
> > Bikram Chatterjee
> > Senior Engineer
> > Alumnus Software Limited
> > Kolkata
> >
> >
> >
> > _______________________________________________
> > lwip-users mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/lwip-users
> >
>
>
> _______________________________________________
>  lwip-users mailing list
>  address@hidden
>  http://lists.nongnu.org/mailman/listinfo/lwip-users
>