[lwip-users] "Simultaneous" close by server & client causes memp_free error

[Dave Harper]

Configuration:
lwIP v1.3.2
FreeRTOS v6.0.4
AT91SAM7X512 based homebrew hardware

Brief Description:
An embedded system running the above software to create (among other things)
a webServer that can service multiple browser clients supporting AJAX
connections.

Problem:
The initial version of my project used the socket interface into lwIP and
worked quite well.  Recently, I decided to add system level trace logging
between the embedded hardware and a PC that would track debug information
from the SAM7 software along with debug information from 18 remote units
communicating with the SAM7 over ZigBee links.  At this point I decided to
switch from the lwIP socket interface to the raw interface.  In doing so I
have stumbled across a problem in which a PCB is released back into the memp
pool twice.  This can happen anywhere from immediately after a browser first
connects or after the system has run for quite a while doing thousands of
transactions.  I suspect I'm doing something wrong but I'm not sure what.  

Basically, after completing a transaction with the browser, the server
software closes the connection by calling tcp_arg(), tcp_sent() and
tcp_recv().  Following this is a loop that will call tcp_close() and keep
doing so if it gets a return status of ERR_MEM.  The PCB state at this point
is ESTABLISHED.  The tcp_close() function executes the "tcp_send_ctrl(pcb,
TCP_FIN)" statement but, in the failing situation, this returns an ERR_MEM
error from tcp_enqueue().  This happens several times.  In the meantime, an
input from the browser results in tcp_input() determining that the
connection has been closed so it calls tcp_pcb_remove(), which results in
the state being set to CLOSED, and then deallocates the PCB.  Shortly after
this the tcp_close() executes again but this time, since the state is now
CLOSED it executes that case statement.  The CLOSED case calls memp_free()
and ends up releasing the same PCB back to the pool that was earlier
released by tcp_input().

This seems like such an obvious case that, if it were a bug in lwIP, many
others would have tripped over it long before I came along.  Therefore I'm
likely doing something stupid but I'm not sure what.  I could add a check in
my close function that checks the PCB state before calling tcp_close() and
not do it if the connection is already closed.  The problem I see with this
in a preemptive system is that the state could change after I've done the
check and before tcp_close() is actually entered.  Further, the comments in
the tcp_close() CLOSED case indicate that closing a PCB that is already in
the CLOSED state is a legitimate thing to do.  Another thing I could do is
to modify either tcp_close() or memp_free() to chase the MEMP_TCP_PCB chain
and see if I'm about to release a PCB that is already in the free chain and
simply return an ERR_OK if this is the case.  While this might fix this
particular problem it does not address what I'm doing wrong in the first
place.

I've been struggling with this issue for a couple of weeks now and am at a
point where I need the guidance of someone more knowledgable about lwIP than
me.  If anyone has any thoughts on this I would really appreciate hearing
them.

Regards,
Dave
-- 
View this message in context: 
http://old.nabble.com/%22Simultaneous%22-close-by-server---client-causes-memp_free-error-tp31251407p31251407.html
Sent from the lwip-users mailing list archive at Nabble.com.