Re: [lwip-devel] Re: [task #7040] Work on tcp_enqueue

[Kieran Mansley]

On Mon, 2009-02-02 at 17:55 +0100, address@hidden wrote:
> Kieran Mansley wrote:
> > On Sun, 2009-02-01 at 13:25 +0100, Jakob Stoklund Olesen wrote:
> >   
> >> Here is a plan:
> >> [..]
> >>     
> > Lets do these separately and one at a time.  I'm quite conservative
> > about changes in the core stack, particularly if they're changing the
> > API.  The move to split tcp_enqueue into two sounds OK as long as we
> > don't end up with much bigger code, and I'm happy to see a neat change
> > to avoid half-filled pbufs.  I'm not sure what all the checksumming
> > changes will give us though.
> >   
> 
> We should decide what's our goal here: When I first started task #7040, 
> I just wanted to make sure lwIP uses full segments where possible, 
> regardless of the length of data tcp_write is called. To reach that, we 
> would first make the simple change to create the first pbuf with such a 
> length that it fits the last segment (if that is not full). I would even 
> see that as a bug-fix, since it has the possibility to be a large 
> performance hole.

Agreed.

> The next thing to do would be the change to allow writing pbufs, not 
> data pointers, to allow zero copy for raw API users. That's a new 
> feature, obviously, but I think the changes for that are minimal: create 
> the pbufs to be enqueued in tcp_write, let tcp_enqueue (which is not an 
> external API function) take pbufs, add a tcp_write_pbuf and we're done.

I'd be persuaded by this if there was a significant number of people who
need this API.  

> The third stage would be to find a way to prevent pbuf queues for 
> multiple small writes: at least one MAC I used was not able to do 
> scatter-gather, which means although I had a DMA-enabled MAC, I had to 
> use memcpy to get the packets in one piece -> large, unnecessary 
> performance hole! This could however require the exact opposite of the 
> previous point (not splitting up tcp_write and tcp_enqueue).

So we really need to know which is the bigger problem for people using
lwIP.  I want to avoid going down one road to solve one problem and then
finding out we've made things worse for another problem.

> The checksum-on-copy changes are totally independent from this but could 
> benefit from the API changes. Even moving checksumming from 
> tcp_output_segment to tcp_enqueue could make a difference since we could 
> benefit from data caches. But still, it has not much to do with the 
> above changes and thus could (and should) wait until these are done and 
> stable, I guess.

Yes - lets put this last one on hold and see what we end up with after
the other changes.

Kieran