[lwip-users] local connection failed on loopif, race?

[Tai-hwa Liang]

Hi,

  Is there anyone having success on using loopif in a multi-threaded
application?  The attached is a modified unix/proj/unixsim/simhost.c.
Basically it is a self-contained server/client which binds/connects to
127.0.0.2 through loopif.

  The problem is that this program failed to run indefinitely and hung
after a few client <-> server transactions(lwIP CVS version, with
LWIP_HAVE_LOOPIF = 1, running on FreeBSD-CURRENT):

freebsd> ./simhost -d
Host at 127.0.0.2 mask 255.0.0.0 gateway 127.0.0.1
System initialized.
TCP/IP initialized.
netif_set_ipaddr: netif address being changed
netif: IP address of interface API message 0x8065a28
cli connecting...
tcpip_thread: API message 0x8065a28
tcp_bind: bind to port 80
tcpip_thread: API message 0x8065a3c
tcp_connect to port 80
tcp_enqueue(pcb=0x8065300, arg=0x0, len=0, flags=2, copy=0)
tcp_enqueue: queuelen: 0
tcp_enqueue: queueing 6509:6510 (0x2)
tcp_enqueue: 1 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6509, ack 6509
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6509, ack 6509, i 0
tcp_output_segment: rtseq 6509
tcp_output_segment: 6509:6509
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        0      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc7     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcpip_thread: API message 0x8065a3c
srv listening return, proceed to accept
tcp_slowtmr: processing active pcb
tcp_slowtmr: processing active pcb
tcp_slowtmr: processing active pcb
tcp_slowtmr: processing active pcb
tcp_slowtmr: polling application
tcp_output: nothing to send (0x0)
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, seg == NULL, ack 6509
tcp_slowtmr: processing active pcb
tcp_slowtmr: processing active pcb
tcp_slowtmr: rtime 6 pcb->rto 6
tcp_slowtmr: cwnd 1024 ssthresh 2048
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 0, seq 6509, ack 6509
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 0, seq 6509, ack 6509, i 0
tcp_output_segment: rtseq 6509
tcp_output_segment: 6509:6509
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        1      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc6     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcpip_thread: IP packet 0x8065a3c
ip_input: iphdr->dest 0x200007f netif->ip_addr 0x200007f (0x7f, 0x7f, 0x2000000)
ip_input: packet accepted on interface lo
ip_input: 
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        1      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc6     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
ip_input: p->len 44 p->tot_len 44
TCP header:
+-------------------------------+
|     4097      |       80      | (src port, dest port)
+-------------------------------+
|           0000006509          | (seq no)
+-------------------------------+
|           0000000000          | (ack no)
+-------------------------------+
|  6 |   |000010|      8096     | (hdrlen, flags (SYN ), win)
+-------------------------------+
|    0x5278     |         0     | (chksum, urgp)
+-------------------------------+
tcp_input: packed for LISTENing connection.
TCP connection request 4097 -> 80.
tcp_enqueue(pcb=0x80653a4, arg=0x0, len=0, flags=12, copy=0)
tcp_enqueue: queuelen: 0
tcp_enqueue: queueing 6516:6517 (0x12)
tcp_enqueue: 1 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6516, ack 6516
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6516, ack 6516, i 0
tcp_output_segment: rtseq 6516
tcp_output_segment: 6516:6516
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        2      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc5     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcpip_thread: IP packet 0x8065a3c
ip_input: iphdr->dest 0x200007f netif->ip_addr 0x200007f (0x7f, 0x7f, 0x2000000)
ip_input: packet accepted on interface lo
ip_input: 
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        2      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc5     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
ip_input: p->len 44 p->tot_len 44
TCP header:
+-------------------------------+
|       80      |     4097      | (src port, dest port)
+-------------------------------+
|           0000006516          | (seq no)
+-------------------------------+
|           0000006510          | (ack no)
+-------------------------------+
|  6 |   |010010|      8096     | (hdrlen, flags (SYN ACK ), win)
+-------------------------------+
|    0x38f3     |         0     | (chksum, urgp)
+-------------------------------+
+-+-+-+-+-+-+-+-+-+-+-+-+-+- tcp_input: flags SYN ACK 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+
State: SYN_SENT
SYN-SENT: ackno 6510 pcb->snd_nxt 6510 unacked 6509
tcp_process: SYN-SENT --queuelen 0
tcp_output: nothing to send (0x0)
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, seg == NULL, ack 6510
State: ESTABLISHED
cli writing HTTP request....
netconn_write: writing 29 bytes (0)
tcpip_thread: API message 0x8065a3c
tcp_write(pcb=0x8065300, arg=0x80623f0, len=29, copy=0)
tcp_enqueue(pcb=0x8065300, arg=0x80623f0, len=29, flags=0, copy=0)
tcp_enqueue: queuelen: 0
tcp_enqueue: queueing 6510:6539 (0x0)
tcp_enqueue: 2 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 29, seq 6510, ack 6510
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 29, seq 6510, ack 6510, i 0
tcp_output_segment: 6510:6539
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        69     | (v, hl, tos, len)
+-------------------------------+
|        3      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dab     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()cli deleting connection
tcpip_thread: API message 0x8065a3c
tcp_close: closing in State: ESTABLISHED
tcp_enqueue(pcb=0x8065300, arg=0x0, len=0, flags=1, copy=1)
tcp_enqueue: queuelen: 2
tcp_enqueue: queueing 6539:6540 (0x1)
tcp_enqueue: 3 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 29, seq 6539, ack 6510
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 29, seq 6539, ack 6510, i 0
tcp_output_segment: 6539:6539
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        40     | (v, hl, tos, len)
+-------------------------------+
|        4      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc7     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcpip_thread: IP packet 0x8065a3c
ip_input: iphdr->dest 0x200007f netif->ip_addr 0x200007f (0x7f, 0x7f, 0x2000000)
ip_input: packet accepted on interface lo
ip_input: 
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        69     | (v, hl, tos, len)
+-------------------------------+
|        3      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dab     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
ip_input: p->len 69 p->tot_len 69
TCP header:
+-------------------------------+
|     4097      |       80      | (src port, dest port)
+-------------------------------+
|           0000006510          | (seq no)
+-------------------------------+
|           0000006517          | (ack no)
+-------------------------------+
|  5 |   |011000|      8096     | (hdrlen, flags (PSH ACK ), win)
+-------------------------------+
|    0x8d1b     |         0     | (chksum, urgp)
+-------------------------------+
+-+-+-+-+-+-+-+-+-+-+-+-+-+- tcp_input: flags PSH ACK 
-+-+-+-+-+-+-+-+-+-+-+-+-+-+
State: SYN_RCVD
TCP connection established 4097 -> 80.
State: ESTABLISHED
tcp_receive: window update 8096
tcp_receive: slow start cwnd 1025
tcp_receive: ACK for 6517, unacked->seqno 6516:6517
tcp_receive: removing 6516:6517 from pcb->unacked
tcp_receive: queuelen 1 ... 0 (after freeing unacked)
tcp_receive: pcb->rttest 6 rtseq 6516 ackno 6517
tcp_receive: experienced rtt 0 ticks (0 msec).
tcp_receive: RTO 5 (2500 milliseconds)
tcp_output: nothing to send (0x0)
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, seg == NULL, ack 6517
State: ESTABLISHED
cli (re)creating connection...
srv receiving HTTP request ....
tcpip_thread: API message 0x8065a3c
tcpip_thread: API message 0x8065a28
tcp_output: sending ACK for 6539
ip_output_if: lo0
IP header:
+-------------------------------+
cli connecting...
| 4 | 5 |  0x00 |        40     | (v, hl, tos, len)
+-------------------------------+
|        5      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc6     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcp_recved: recveived 29 bytes, wnd 8096 (0).
tcpip_thread: API message 0x8065a3c
tcp_connect to port 80
tcp_enqueue(pcb=0x806525c, arg=0x0, len=0, flags=2, copy=0)
tcp_enqueue: queuelen: 0
tcp_enqueue: queueing 6527:6528 (0x2)
tcp_enqueue: 1 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6527, ack 6527
tcp_output: snd_wnd 8096, cwnd 1, wnd 1, effwnd 0, seq 6527, ack 6527, i 0
tcp_output_segment: rtseq 6527
tcp_output_segment: 6527:6527
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        44     | (v, hl, tos, len)
+-------------------------------+
|        6      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc1     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()netconn_recv: received 0x806571c (err 0)
srv: it's a GET method GET /index.html HTTP/1.0


srv delete server connection
tcpip_thread: API message 0x8065a3c
tcp_close: closing in State: ESTABLISHED
tcp_enqueue(pcb=0x80653a4, arg=0x0, len=0, flags=1, copy=1)
tcp_enqueue: queuelen: 0
tcp_enqueue: queueing 6517:6518 (0x1)
tcp_enqueue: 1 (after enqueued)
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 0, seq 6517, ack 6517
tcp_output: snd_wnd 8096, cwnd 1024, wnd 1024, effwnd 0, seq 6517, ack 6517, i 0
tcp_output_segment: rtseq 6517
tcp_output_segment: 6517:6517
ip_output_if: lo0
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        40     | (v, hl, tos, len)
+-------------------------------+
|        7      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc4     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
netif->output()tcpip_thread: IP packet 0x8065a3c
ip_input: iphdr->dest 0x200007f netif->ip_addr 0x200007f (0x7f, 0x7f, 0x2000000)
ip_input: packet accepted on interface lo
ip_input: 
IP header:
+-------------------------------+
| 4 | 5 |  0x00 |        40     | (v, hl, tos, len)
+-------------------------------+
|        5      |010|       0   | (id, flags, offset)
+-------------------------------+
|  255  |    6  |    0x7dc6     | (ttl, proto, chksum)
+-------------------------------+
|  127  |    0  |    0  |    2  | (src)
+-------------------------------+
|  127  |    0  |    0  |    2  | (dest)
+-------------------------------+
ip_input: p->len 40 p->tot_len 40
TCP header:
+-------------------------------+
|       80      |     4097      | (src port, dest port)
+-------------------------------+
|           0000006517          | (seq no)
+-------------------------------+
|           0000006539          | (ack no)
+-------------------------------+
|  5 |   |010000|      8096     | (hdrlen, flags (ACK ), win)
+-------------------------------+
|    0x4edf     |         0     | (chksum, urgp)
+-------------------------------+
+-+-+-+-+-+-+-+-+-+-+-+-+-+- tcp_input: flags ACK -+-+-+-+-+-+-+-+-+-+-+-+-+-+
State: FIN_WAIT_1
tcp_receive: window update 8096
tcp_receive: slow start cwnd 2048
tcp_receive: ACK for 6539, unacked->seqno 6510:6539
tcp_receive: removing 6510:6539 from pcb->unacked
tcp_receive: queuelen 3 ... 1 (after freeing unacked)
tcp_receive: pcb->rttest 6 rtseq 6509 ackno 6539
tcp_receive: experienced rtt 0 ticks (0 msec).
tcp_receive: RTO 5 (2500 milliseconds)
tcp_output: nothing to send (0x0)
tcp_output: snd_wnd 8096, cwnd 2048, wnd 2048, seg == NULL, ack 6539
State: FIN_WAIT_1
srv listening return, proceed to accept
[...hanging...]

simhost.c
Description: Text document