[lwip-devel] [task #6735] Provide new pbuf type: PBUF_RAM_NOCOPY

[Jonathan Larmour]

Follow-up Comment #10, task #6735 (project lwip):

Re Frederic's comment #9:
Oh, I knew Kieran had been marking some items as for 1.3, but I hadn't
realised that there was a feature freeze now - I must have missed that, and
there still seemed quite a bit of activity.

Even though it by rights shouldn't change existing behaviour, if it has to
wait, it has to wait. In that case, I'll hold off committing, and making the
second patch. 

Still, if people have comments, they're still welcome.

> if adding this new option don't make mandatory to change 
> ethernet drivers, so, an option is not necessary to my point of
> view... I see than mainly like an optimization possible for 
> netconn api/raw > api users, I am wrong?) 

Certainly that was my original intention - it's not used till someone creates
a pbuf of that type. However, I remember Simon bringing up some situations
within the stack when it would be useful for the stack itself to create them.
That's for the future though.

Jared wrote in comment #8:
> I wrote my Ethernet driver completely zero-copy and I haven't
> had any issues yet -- what part of lwIP code expects to be able
> to alter a pbuf after it is submitted? (Maybe I haven't enabled
> that module yet!) 

One problem is the user. As Kieran wrote in bug 11400: "the API does
currently seem to give the application the right to do what it likes with the
pbuf, so I think we will have to copy if we can't send it immediately."

The most obvious example is if someone sends a UDP packet. So e.g. they call
lwip_sendto in sockets.c, which makes a PBUF_REF (via calling netbuf_ref). It
then sends it with netconn_send, which eventually goes through the stack until
the linkoutput calls into the driver. That returns immediately back up through
the stack, but if the driver hasn't really sent the packet data, but has
queued it or is half-way through sending it, the user has the freedom to start
changing the original data they passed in.

But the same holds for PBUF_RAM UDP packets with the netconn API - you can
e.g. reuse or chain a netbuf after having just sent data from it. 

> A brief glance through the code seems to
> suggest that "some_output (p); pbuf_free(p);" is a common
> sequence of code. To require a driver to allocate a new buffer,
> copy the data, and then transmit seems to be a waste when the
> next statement upon return is often a pbuf_free anyway.

If you think the driver will be transmitting that fast, you could always
instead ensure the driver doesn't return until it _is_ sent.

But otherwise, there is a race condition. The idea is that if your usage
pattern is something like:
p = pbuf_alloc(PBUF_RAM, ...);
fill_p_with_data(p);
some_output(p);
pbuf_free(p);

then instead since you know you're going to free the data afterwards, this
would become
p = pbuf_alloc(PBUF_RAM_NOCOPY, ...);
fill_p_with_data(p);
some_output(p);

So does this make the purpose clearer? It's possible that zero copy can work
if you happen to have fast enough ethernet hardware, and depending on the
driver design (e.g. how much it can queue), and the application. But there are
no guarantees in general. It's likely that the more Frederic and Simon improve
the sockets.c performance :-), the more likely it is that this problem will
show up. It should already be fairly easy for it to show up with the netconn
API.

TCP is a bit different because there's a char* and length, and a copy flag
passed in, instead of a pbuf. The copy flag can be 1, in which case it's
copied. (Although the hardware driver won't know that it's safe to use it
without copying). The only situation in which the copy flag can be 0 is if
you're never prepared to change the data.

    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/task/?6735>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.nongnu.org/