[lwip-devel] [bug #43779] select(), close(), and TCP retransmission error

[Jonathan Portwood]

URL:
  <http://savannah.nongnu.org/bugs/?43779>

                 Summary: select(), close(), and TCP retransmission error
                 Project: lwIP - A Lightweight TCP/IP stack
            Submitted by: jpbears1
            Submitted on: Mon 08 Dec 2014 02:33:17 PM GMT
                Category: sockets/netconn
                Severity: 3 - Normal
              Item Group: Faulty Behaviour
                  Status: None
                 Privacy: Public
             Assigned to: None
             Open/Closed: Open
         Discussion Lock: Any
         Planned Release: 
            lwIP version: 1.4.1

    _______________________________________________________

Details:

We ran across a fairly esoteric issue with the LwIP socket implementation.

The issue stems from using a TCP socket as a server. If the client sends data
to the server and is done, the client sends a "FIN" with the last packet. As
with all network stacks, in LwIP this results in select() notifying the
application of data on the socket and then a EOF on the server side draining
the socket. This is the correct behavior. In this situation you see this in
the debug:

State: ESTABLISHED
tcp_receive: pcb->rttest 0 rtseq 6537 ackno 6538
tcp_receive: received FIN.
State: CLOSE_WAIT


However,  in a very specifically crafted case where the packet right before
the FIN came from the client was dropped, a TCP retransmission must occur. 
The FIN packet was received and queued. Then the retransmitted packet is
received, and the FIN is dequeued. You see this in the debug log:

State: ESTABLISHED
tcp_receive: pcb->rttest 0 rtseq 657 ackno 6538
tcp_receive: dequeued FIN.
State: CLOSE_WAIT

In this situation, the socket layer will notify select() and the server app
will read the socket.  In this situation, the last bytes of the transfer are
delivered to the application, but not the EOF.  The server side never receives
a direct notification of the EOF.
In the sockets.c file its this function: lwip_recvfrom().  A FIN results in a
NULL msg sent to the netconn.
This results in this call returning ERR_CLSD:

 if (netconn_type(sock->conn) == NETCONN_TCP) {
        err = netconn_recv_tcp_pbuf(sock->conn, (struct pbuf **)&buf);

You see this in the log:
lwip_recvfrom: netconn_recv err=-12, netbuf=0

However, then below:
      if (err != ERR_OK) {
        if (off > 0) {
            ...

If off is greater than 0 then the leftover data is sent to the application and
returned. The knowledge of the error (in this case CLOSED) is lost, and
select() is never rewoken on the socket.  The workaround I have come up with
is if this happens, grab a lock and increment the rcvevent for the socket so
that select() will wake up the thread one more time and the application will
recall lwip_recvfrom() and this time it will grab the EOF:

      if (err != ERR_OK) {
        if (off > 0) {
          /* update receive window */
          netconn_recved(sock->conn, (u32_t)off);
          /* already received data, return that */
          sock_set_errno(sock, 0);
          // workaround begin
          SYS_ARCH_PROTECT(lev);
          sock->rcvevent++; /* signal one more receive event for NULL */
          SYS_ARCH_UNPROTECT(lev);
          // workaround end
          return off;
        }

Again, this has to be a specific dropped packet - we ran across this because
of a MAC layer issue causing that particular packet to be dropped.  It appears
that if any other packet in the transmission is dropped (including the last
packet) than there are no issues.




    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/bugs/?43779>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/