Re: [lwip-users] netconn_write throughput stuck at 1024 bytes

[Kieran Mansley]

On Wed, 2009-01-21 at 07:18 -0800, DownyTif wrote:
> >DownyTif wrote:
> >> 
> >> first, I want to say that I'm new to this lwip world. I'm trying to make
> >> a
> >> Ethernet speed test using the EVK1100 board from ATMEL (AVR32). The
> >> communication will be between the board and a PC using 2 ports, one
> >> dedicated to bidir requests/responses between the board and the host, the
> >> other dedicated to maximum speed data sending from the board to the host.
> >> I
> >> would like to be able to send packets of 32768 bytes.
> 
> >Do you really mean "packets" there? Is this ethernet? Ethernet's normal 
> >MTU is around 1500 bytes, unless you're going to be using jumbo frames. 
> >You can send larger IP packets, but they get fragmented down to the MTU 
> >size, which is inefficient.
> 
> Yeah, I meant Ethernet frames. I didn't know about that MTU... So what would
> help me achieve the maximum throughput? On another system using the same
> host but a different board, I was able to get about 65Mbps of throughput
> sending Ethernet frames of ~450kb each (lot of data). 

No you didn't!  Ethernet frames are usually limited to 1500 bytes each,
sometimes up to 9000 bytes if jumbo frames are configured.  IP also has
a maximum datagram size (65535 bytes) and TCP has a maximum segment
length (typically 1460 bytes, sometimes more if you're running on jumbo
Ethernet frames).  These limits are a *lot* less than 450KB.  I think
you misunderstand how API-level writes are translated into Ethernet
frames.   

> Ok, that other board
> is using a in-house IP stack (I don't have the code) and a dedicated
> Ethernet MCU, so I don't expect to get those results with my AVR32 board.
> But still, I'm looking for the fastest transfers. If I understand correctly,
> when I "netconn_write()" a frame of let's say 4096 bytes and my configured
> MSS is 1460, the lwip will divide my frame into 3 segments (1460, 1460 and
> 284) and send to the host those 3 frames in one send right? Or will the lwip
> send 3 frames in 3 send actions resulting in many ACK hand-shaking? In other
> words, is there a difference in this:
> 
> netconn_write(..., 4096, ...);
> 
> or
> 
> netconn_write(..., 1460, ...);
> netconn_write(..., 1460, ...);
> netconn_write(..., 284, ...);

The correct answer is "maybe".  It depends on what traffic there is in
flight, how much congestion there is in the network, if there has been
any data loss, what buffers are available, how fast the other end
acknowledges your packets, how fast the other end consumes the data,
etc.  In short, the TCP stack can segment and combine your
netconn_writes however it likes, and you can't accurately predict what
the Ethernet frames will look like.  You can say that no single frame
will be greater in size than the TCP MSS, IP datagram size, or ethernet
MTU of either end that is communicating, and if you go over a larger
network your packet may even be segmented by the network into multiple
smaller ones.

With UDP it's a bit different. 

> I traced the code and you are right. I found that until I accept the
> connection, my mss is 2048, but the incoming connection puts it to 1460.
> Unfortunatly, I don't think I can change that on the host, because it's a
> Windows PC and I'm not sure I want to mess with those settings on the
> client's computers.

Think of it this way: your MSS setting says "I can handle packets up to
2048 bytes", and the Windows PC replies "I can only handle packets up to
1460 bytes".  The stack therefore uses the smaller value of the two, as
both ends need to agree on a value.  This is entirely normal TCP
behaviour though, and nothing special about lwIP.

Kieran