Re: [lwip-users] TCP spurious Retransmission and Dup Ack issue

[Axel Lin]

2017-01-15 23:20 GMT+08:00 Axel Lin <address@hidden>:
> 2017-01-09 21:56 GMT+08:00 Sergio R. Caprile <address@hidden>:
>> Your sequence number is jumping backwards.
>> Most common causes are either one of your apps is trashing memory of you
>> have a bad DMA driver.
>
> Hi Sergio,
>
> Thanks for your comments.
> It seems every time before run into tcp-out-of-order issue, the code
> run in below path:
> I'm wondering if anything unusual in below code path?
>
> Note, I have LWIP_NETIF_TX_SINGLE_PBUF set, so TCP_WRITE_FLAG_COPY is set:
> I also add some run-time value of related variables in below:

I have tried many times and found that the symptom is consistent:
Every time I run into below code path, i.e.  when pcb->unsent_oversize is 0,
I run into the tcp-out-of-order issue.

>
> In tcp_write():
>
>  468     oversize = pcb->unsent_oversize;
> //oversize=0, pcb->unsent_oversize=0
>  469     if (oversize > 0) {
>  470       LWIP_ASSERT("inconsistent oversize vs. space",
> oversize_used <= space);
>  471       seg = last_unsent;
>  472       oversize_used = LWIP_MIN(space, LWIP_MIN(oversize, len));
>  473       pos += oversize_used;
>  474       oversize -= oversize_used;
>  475       space -= oversize_used;
>  476     }
>  477     /* now we are either finished or oversize is zero */
>  478     LWIP_ASSERT("inconsistend oversize vs. len", (oversize == 0)
> || (pos == len));
>  479 #endif /* TCP_OVERSIZE */
>  480
>  481     /*
>  482      * Phase 2: Chain a new pbuf to the end of pcb->unsent.
>  483      *
>  484      * As an exception when NOT copying the data, if the given data 
> buffer
>  485      * directly follows the last unsent data buffer in memory,
> extend the last
>  486      * ROM pbuf reference to the buffer, thus saving a ROM pbuf 
> allocation.
>  487      *
>  488      * We don't extend segments containing SYN/FIN flags or options
>  489      * (len==0). The new pbuf is kept in concat_p and pbuf_cat'ed at
>  490      * the end.
>  491      */
>  492     if ((pos < len) && (space > 0) && (last_unsent->len > 0)) {
> // pos=0, len=58, space=1148, last_unsent->len=4
>  493       u16_t seglen = space < len - pos ? space : len - pos;
> // seglen=58
>  494       seg = last_unsent;
>  495
>  496       /* Create a pbuf with a copy or reference to seglen bytes. We
>  497        * can use PBUF_RAW here since the data appears in the middle of
>  498        * a segment. A header will never be prepended. */
>  499       if (apiflags & TCP_WRITE_FLAG_COPY) {
>  500         /* Data is copied */
>  501         if ((concat_p = tcp_pbuf_prealloc(PBUF_RAW, seglen,
> space, &oversize, pcb, apiflags, 1)) == NULL) {     // oversize
> becomes 1090 here
>  502           LWIP_DEBUGF(TCP_OUTPUT_DEBUG | LWIP_DBG_LEVEL_SERIOUS,
>  503                       ("tcp_write : could not allocate memory for
> pbuf copy size %"U16_F"\n",
>  504                        seglen));
>  505           goto memerr;
>  506         }
>  507 #if TCP_OVERSIZE_DBGCHECK
>  508         last_unsent->oversize_left += oversize;
>  509 #endif /* TCP_OVERSIZE_DBGCHECK */
>  510         TCP_DATA_COPY2(concat_p->payload, (const u8_t*)arg + pos,
> seglen, &concat_chksum, &concat_chksum_swapped);
>  511 #if TCP_CHECKSUM_ON_COPY
>  512         concat_chksummed += seglen;
>  513 #endif /* TCP_CHECKSUM_ON_COPY */
>  514         queuelen += pbuf_clen(concat_p);
>
> And then I got below message latter after above code path.
> tcp_write: too long queue 24 (max 24)
>
> I also upload the wireshark log in below link for reference:
> https://www.dropbox.com/s/16slydot7rx221f/lwip_tcp_out_of_order.pcapng?dl=0
> lwip device is running at 192.168.0.101 8080 port.
>
> Thanks,
> Axel