Re: [libmicrohttpd] winsock & EPOLL_TURBO

[LRN]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10.08.2013 19:39, Christian Grothoff wrote:
> On 08/10/13 15:46, LRN wrote:> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 10.08.2013 00:24, Christian Grothoff wrote:
>>> On 08/09/13 20:59, LRN wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>>>
>>>> On 08.08.2013 23:47, LRN wrote:
>>>>> On 08.08.2013 22:47, Christian Grothoff wrote:
>>>>>> On 08/08/13 20:22, LRN wrote:
>>>>>>> I've tried many different wait time values (from 50ms to
>>>>>>> 1000ms), different waiting techniques (busylooping until
>>>>>>> enough time passes, Sleep(), WaitForSingleObject()), also
>>>>>>> tried to enable curl debugging.
>>>>>>>
>>>>>>> When curl debugging is enabled, the test often passes, but
> sometimes it
>>>>>>> fails. When curl debugging is not enabled, it always fails,
>>>>>>> no matter
> how small
>>>>>>> the wait time is.
>>>>>>>
>>>>>>> Thing is, curl does send the data after the delay, and MHD
>>>>>>> does
> receive
>>>>>>> data, and sends HTTP status 500 - but then either MHD drops
>>>>>>> the connection prematurely, or curl fails to read from it.
>>>>>>> I have no idea what is happening.
>>>>>
>>>>>> Odd. have you tried MHD's "demo" application (which has a
>>>>>> POST) on W32 ever (with a browser)? This could be, after all,
>>>>>> an issue with curl, so this way we'd at least know that POST
>>>>>> works in principle...
>>>>> Will try.
>>>> Ended up gdb'ing through curl.
>>>>
>>>> Found the exact WinSock error code that it gets, googled it up,
>>>> came up wtih [1]. Sounds very much like our case.
>>>>
>>>> [1] http://www.chilkatsoft.com/p/p_299.asp
>>>
>>> Hmm. That's strange, because the request is not ill-formed and I
>>> don't see why MHD would do a partial read and thus trigger this.
>>>
>>> It also is unclear to me why the 'sleep' that we add would cause
>>> such a winsock error. Also, if [1] is right, then Winsock would
>>> really not implement TCP correctly, as while the data from the
>>> sender might be "lost", the other direction of the TCP stream
>>> should not be affected (by my reading of how TCP/IP is _supposed_
>>> to be implemented).
>>>
>>> You might try adding the new 'MHD_USE_EPOLL_TURBO' flag to see if
>>> that helps; in this context, it would disable the 'shutdown' calls
>>> on the TCP socket by MHD, which might resolve this IF winsock is
>>> really broken.
>> MHD_USE_EPOLL_TURBO solves the problem.
>>
>> No one ever said that WinSock is 100% TCP-compliant... :(
> 
> 
> On 08/10/13 16:05, LRN wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 10.08.2013 17:49, Christian Grothoff wrote:
>>> On 08/10/13 15:46, LRN wrote:
>>>>>
>>>>> Hmm.  That's strange, because the request is not ill-formed and I don't
>>>>> see why MHD would do a partial read and thus trigger this.
>>>>>
>>>>> It also is unclear to me why the 'sleep' that we add would cause such a
>>>>> winsock error.  Also, if [1] is right, then Winsock would really not
>>>>> implement TCP correctly, as while the data from the sender might be
>>>>> "lost", the other direction of the TCP stream should not be affected (by
>>>>> my reading of how TCP/IP is _supposed_ to be implemented).
>>>>>
>>>>> You might try adding the new 'MHD_USE_EPOLL_TURBO' flag to see if that
>>>>> helps; in this context, it would disable the 'shutdown' calls on the TCP
>>>>> socket by MHD, which might resolve this IF winsock is really broken.
>>>> MHD_USE_EPOLL_TURBO solves the problem.
>>>>
>>>> No one ever said that WinSock is 100% TCP-compliant... :(
>>>
>>> Wow. Now that's insane.  I guess we'll need to make that then the
>>> default, at least for W32...
>>>
>>
>> After thinking a bit, here's a possible explanation:
>> There's some overhead when curl connects to MHD (MHD has to accept
>> connection, do stuff, etc). If curl manages to fill the socket buffer on
>> its side right after connecting, then by the time MHD responds, curl is
>> ready to read the response before the connection is shut down by Winsock.
>>
>> That's my theory, anyway. Difficult to say exactly what happens with
>> that proprietary API...
> 
> I've pushed a change that forces EPOLL_TURBO for W32 to SVN HEAD.
> 
> cc'ing mailinglist to archive the discussion...
> 

I've made a patch that fixes the problem, no need to disable shutdown()
completely. Here it is.

The problem is that winsock freaks out (and prevents you from sending
the reply) if you shutdown with SD_RECEIVE while there is data in recv
buffer.

Sorry, couldn't send the patch sooner - had to reproduce the problem in
a testcase, then test the patch in MHD.

- -- 
O< ascii ribbon - stop html email! - www.asciiribbon.org
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (MingW32)

iQEcBAEBAgAGBQJSBmE4AAoJEOs4Jb6SI2Cwk8wIAKL5HRVjG3b6qzHn0sdtR6Mm
0ucNs5nhmvV9JQ8fMC1JBMZ4Dv26k0QIQEtblE9OgHpNzZXi+f6JIR7wsvBpMjRe
4BjA2kP5nI/2ppDM/EFOt4KZ2GKnMZNK4z9/GPy/J1gb5ECa8uaa7nxo5M+yAgrM
BILwUqwLJKW45FGEfdYn+KsrQlTrGdrwKR3aPNGIGm2DS2MXxatt2tbQ/hchT8Go
3dqvPFoRrbpQpNL5Olvu8Ot4bOR4pBcPq2skIeolfNYmcMj3nDiVNMBZXNN/3Z2k
SuQp9tFyMFqTuQhZ0DenMQ9JEtgebQB3HqITfUVng6Loo6pmjsuv3+6RchQgGVA=
=b+DS
-----END PGP SIGNATURE-----

z0003-empty-recv-buffer.mingw32.patch
Description: Text document