Re: [libmicrohttpd] Some issues with MHD

[Olivier Delhomme]

On Fri, Nov 06, 2015 at 10:09:51PM +0100, Olivier Delhomme wrote:
> On Wed, Nov 04, 2015 at 11:00:42PM +0100, Christian Grothoff wrote:
> > Hmm. That's not even clearly indicative of an MHD-related issue.
> 
> I'm even not sure that I'm doing things the right way... 
> 
> > What you should look at is the buffer size MHD offers when calling
> > recv() on the TCP stream.  The size of the buffer MHD offers to the
> > application can be smaller simply because the OS didn't receive 13k data
> > chunks between calls anymore -- be it because the sender is slower, TCP
> > had a congestion event or your receiver happens to be draining the
> > buffer slightly faster. So there are many _other_ reasons that could
> > explain a decrease in upload_buffer_size.
> > 
> > So to diagnose this better, you first should check if the cause is
> > really the size of the IO buffer offered by MHD to the kernel. Given
> > that this is all for the same connection, I doubt this is the cause ---
> > unless your application again doesn't always handle 100% of the data in
> > each call, then you could of course also see such patterns emerge.
> 
> I'm sorry but I'm not sure to understand what your are saying as I'm
> not an expert in TCP nor network related kernel things.
> 
> To diagnose it I managed to code a minimalistic server (160 lines with
> comments) and a minimalistic client (200 lines with comments) that 
> simulates the behavior of my programs. It is published here:
> https://github.com/dupgit/shrinkage
> 
> I found the same behavior regarding the *upload_size value as you can
> see in the *.png images and the file shrinkserver.output. The whole
> run to observe this is about 6-7 minutes. I use the lo interface and
> the throughput is around 80 Mbits/s. It has been tested on my laptop
> that was running gnome desktop with very few opened applications.
> The distribution is a Debian Jessie. shrinkserver is linked with
> libmicrohttpd r36641 and shrinkclient is linked with system's libcurl
> 7.38.0-4+deb8u2.
> 
> Can you tell me if this code leads to something similar with your
> environment ?

I just tried a few things : 

1) run shrinkserver
2) comment the while statement and leave only one POST for one 
   connection and then use a for statement in your shell to execute 
   multiple times the client:
   for i in $(seq 1 1000); do ./shrinkclient; done;

   -> the shrinkage is not hapenning...
3) run shrinkclient with it's while statement until you get the 
   shrinkage...
4) do the same as in 2) and run one or two times manualy and you'll
   see the shrinkage has gone away !

Regards,

Olivier.