One other side effect of our bug was that the socket's rcvevent count
was now out of whack with mailbox. Just an FYI.
Sizing the mailbox to hold all pbufs is a reasonable solution I think
(given the goal of lightweight), in our case we have tons of sockets
(25) -- our application is not so lightweight!
Just an FYI, reducing the TCP_WND is not, however, a completely
foolproof remedy. Our TCP_WND happens to be pretty low (2920) -- this
does throttle the outstanding data on the wire for a single TCP
connection, but does not (directly) effect the amount of data in the
mailbox -- since the window is not tied to the emptying of the mailbox.