[lwip-users] Bug on lwIP TCP section (or HTTP client from contrib)?

[Enrico Murador - Research & Development - CET]

Dear Sirs,

I'm using lwIP from head revision, together with semi-stock HTTP server from contrib head. I've modified it to add basic authentication capabilities, although I think my patch should not be related with the issue.
And please note I'm not so confident with TCP window calculations, so there is a good chance all of this is due entirely to my fault!

The issue is that, during a TCP transaction, client gets a (not necessary, in my opinion) RST message from server after (client) sending a FIN+ACK message.

I've attached a Wireshark log of the TCP sequence. Note that it is not complete, because I was in debug and, after the client's FIN+ACK I got a breakpoint hit (and this is the issue) on a call to tcp_rst() from tcp_close_shutdown() function, because at that point the pcb receive window (rcv_wnd) was "off by one": 2919 instead of 2920 (2920 is TCP_WND_MAX(pcb)). For completeness, at that point rcv_ann_wnd was 2571 (and rcv_ann_right_edge was 3271422395: weird number, hoping there is not another issue on this as well...).

I think the "off-by one" on tcp window was there because the presence of the last FIN flag was not yet considered on window calculation at the time tcp_close() was called by http_close_or_abort_conn() function.

The call stack, at breakpoint point, was this:

tcp_input()
http_sent [tcp_input calls TCP_EVENT_SENT(pcb, 183, 0)]
http_send()
http_eof() [we reached the end of the file]
http_close_conn()
http_close_or_abort_conn()
tcp_close()
...and finally tcp_close_shutdown(). Now, with pcb state == CLOSE_WAIT and rcv_wnd == 2919, so tcp_rst() gets called.

If I'm not wrong, lwIP TCP takes care of the presence of a FIN (for the current receive window calculation) on line 468 of tcp_in.c  file (hoping you haven't changed the file in the last few days), on tcp_input() function. The call to TCP_EVENT_SENT() is before this point.

If I'm not wrong and there is effectively a bug, is it in the way tcp_close() is called from HTTP server? Or is it in TCP stack itself?

Thank you
Enrico

TCP_RST_issue.pcapng
Description: Binary data