Re: [lwip-users] TCP performance in receiving

[Atte Kojo]

K.J. Mansley wrote:
> Thanks for your detailed response, it's much appreciated.  It sounds
> like all the data is getting processed by each stage in batches, and so
> tcp_receive() processes all the received data (and acknowledges it with
> a small window), and later the application processes and opens the
> window, but this "window-update-ACK" doesn't get through till a timer
> fires, as by then the sender has stopped sending.  Is that right?  It
> was this delay between receiving and processing that I was missing.

Yes, this is what I meant.

> At a guess, your application reads a large block (ie. similar to the
> window size) of data in one go from the stack, and calls tcp_recved()
> just once for that large block?  Is that correct?

Something like that. I read data in about 1,5kb chunks, and tcp_recved() 
gets called once for each chunk.

> Obviously, a larger window would make it possible for the sender to keep
> streaming, and then the window updates would get sent promptly, but that
> may be something that the memory limited lwIP users can't afford to do.
>
> It's a tricky problem to solve without causing another problem for
> another data pattern (i.e. duplicate ACKs sent when the application
> processes data in small chunks as fast as the stack will deliver them). 
> Perhaps a threshold in tcp_recved() would help, so that if the window is
> currently very small, it sends an ACK immediately.  If it's larger, and
> there is an ACK already outstanding, it does what it does now.  Do you
> think that would help in your situation?

Erm, no. The original problem is that the acks only get sent with small 
window sizes, so sending acks with very small window sizes doesn't 
actually accomplish anything. My original fix is to send out an extra 
ack only if TF_ACK_DELAYED flag is set and the send window is at 
maximum, like this:

 if (!(pcb->flags & TF_ACK_DELAY) &&
    !(pcb->flags & TF_ACK_NOW)) {
   /*
    * We send an ACK here (if one is not already pending, hence
    * the above tests) as tcp_recved() implies that the application
    * has processed some data, and so we can open the receiver's
    * window to allow more to be transmitted.  This could result in
    * two ACKs being sent for each received packet in some limited cases
    * (where the application is only receiving data, and is slow to
    * process it) but it is necessary to guarantee that the sender can
    * continue to transmit.
    */
   tcp_ack(pcb);
 } else if (pcb->flags & TF_ACK_DELAY && pcb->rcv_wnd == TCP_WND) {
   tcp_ack_now(pcb);
 }

This way a duplicate ack is sent only when the senders window is empty. 
I can't tell how it behaves with small chunks of continuous data, but 
with bursts of large chunks it works pretty well. Someone with a 
different kind of an environment might like to test it also.

Does anyone know how other TCP stacks handle this kind of a situation?

If/Because duplicate acks cause problems with some stacks, the else 
clause could be protected with LWIP_ALLOW_DUPLICATE_ACKS (or 
LWIP_UGLY_TCP_PERFORMANCE_HACK) configuration option that defaults to 
off (and is well documented, of course ;-). These kind of applications 
are probably quite rare so this way the modification wouldn't harm 
anyone not needing it, but would be there for the occasional case where 
it is helpful.

        - Atte

--
3. "She may not look like much, but she's got it where it counts, kid."
        - Top 10 suggestive lines from Star Wars: A New Hope