Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd

[Christian Grothoff]

Hi!

MHD has no expectations as to what your cleanup call should do, other than 
free your own resources so that they don't leak.

Also, the callback function should be called if there is a fresh HTTP request 
in any case, unless you

1) corrupted memory (all bets are off)
2) broke your own external select loop and stopped calling MHD itself
3) badly ran out of resources (i.e. FD limit hit and 'accept'ing the client
    socket fails, or we don't have enough memory for the connection, etc.)

Also, there is only one case (failure to process upload data by client AND 
per-connection buffer full AND  (internal select mode OR thread-per-
connection)) where MHD itself generates an internal server error.  This should 
also only be possible with PUT/POST request (GET's usually don't have upload 
data), so I doubt this applies here.

I suggest you check what you get for your app if you use a 2nd browser after 
the 1st one ran into trouble (still no new call to the callback?).  Also, use 
strace to see if there is actually a 2nd connection being accepted and to find 
out what exactly happened to the first connection (shutdown? close? still 
active?). 

Happy hacking!

Christian

On Monday 03 January 2011 09:33:33 David J Myers wrote:
> Happy New Year!
> 
> Ok, I can start the JPEG stream now and it runs once ok. However if I stop
> the stream from the browser then start it again, the server gives me
> "Internal application error, closing connection."
> It doesn't ever get into my ContentReaderCallback again, until I restart
> the server app.
> The relevant code is
>       response = MHD_create_response_from_callback (MHD_SIZE_UNKNOWN,
> 
> 32 * 1024,     /* 32k page size */
> 
> CHttpWrapper::http_streamer,
> 
> pContext,
> 
> CHttpWrapper::http_free_stream_callback);
>       ret = MHD_add_response_header (response, "Content-Type",
> "multipart/x-mixed-replace; boundary=--myboundary");
>       ret = MHD_queue_response (connection, MHD_HTTP_OK, response);
>       MHD_destroy_response (response);
> 
> Is there something I need to do in my http_free_stream_callback that
> re-enables the callback mechanism?
> 
> Best regards
> David
> 
> -----Original Message-----
> From: Christian Grothoff [mailto:address@hidden
> Sent: 23 December 2010 14:30
> To: David J Myers
> Cc: 'libmicrohttpd development and user mailinglist'
> Subject: Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd
> 
> Hi!
> 
> The answer is kind-of simple: you are NOT going to send a Content-Length
> header to the browser.  If you have a stream of unknown length, HTTP
> specifies
> that it is fine not to send a length header, and that's just what needs to
> happen in this case.  If your browser can handle a stream of images over
> the
> 
> same HTTP stream, it must also deal with the fact that the HTTP won't
> specify
> boundaries between the images.  That's just HTTP, not so much MHD.
> 
> Happy hacking,
> 
> Christian
> 
> On Thursday, December 23, 2010 02:51:40 pm David J Myers wrote:
> > Hi Christian,
> > 
> > I'm still having problems with response headers in my jpeg webserver.
> > 
> > 
> > 
> > If I want to send a single JPEG ie a snapshot, I still need to add my own
> > header for the Content-Length in my ContentReaderCallback, because I
> > don't know how big this image is going to be. However the global headers
> > are already terminated with \r\n\r\n, and any header I insert after this
> > is ignored by the browser. How, in my ContentReaderCallback, can I
> > insert my own headers before the \r\n\r\n? You told me previously that I
> > could not use MHD_add_response_header in the callback.
> > 
> > 
> > 
> > Thanks and regards
> > 
> > David
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > From: Christian Grothoff [mailto:address@hidden
> > Sent: 11 November 2010 18:41
> > To: David J Myers
> > Cc: 'libmicrohttpd development and user mailinglist'
> > Subject: Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd
> > 
> > On Thursday, November 11, 2010 06:41:47 pm David J Myers wrote:
> > > So you don't need to call MHD_queue_response for each frame? Just
> 
> copying
> 
> > > the data into the buffer in the callback is enough to send the data to
> > > the client? What about the inter-frame headers,
> > > --myboundary
> > > Content-Type: image/jpeg
> > > Do I just put these manually in the callback buffer at the start of the
> > > JPEG data? ie. not using MHD_add_reponse_header?
> > 
> > Yep.  MHD_add_response_header is only for the 'global' headers, MHD has
> > no idea about 'inter-frame' headers.
> > 
> > Christian
> > 
> > > Thanks again
> > > David
> > > 
> > > 
> > > -----Original Message-----
> > > From: Christian Grothoff [mailto:address@hidden
> > > Sent: 11 November 2010 17:30
> > > To: David J Myers
> > > Cc: 'libmicrohttpd development and user mailinglist'
> > > Subject: Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd
> > > 
> > > You do NOT create another response object, you simply return the next
> > > frame(s)
> > > the next time you are called.  If the next frame is not (yet) ready,
> > > you return zero (if in 'external select' mode) or simply block (if in
> > > thread-per-
> > > connection mode).
> > > 
> > > Best,
> > > 
> > > Christian
> > > 
> > > On Thursday, November 11, 2010 05:50:48 pm David J Myers wrote:
> > > > Ok, thanks again, Christian.
> > > > Just to get this straight then. In the Access Handler Callback, I
> > > > call 'MHD_create_response_from_callback' with
> > > > MHD_SIZE_UNKNOWN. Then call MHD_add_response_header to add my
> > > > headers.
> > > 
> > > Then
> > > 
> > > > in the Content Reader Callback, I can fill in the JPEG data, but what
> > > 
> > > about
> > > 
> > > > subsequent frames? Do I create another response_from_callback and
> > > > repeat for each frame, or do I return zero from the Content Reader
> > > > Callback so that it calls me again for the same response? Can you add
> > > > headers within the Content Reader Callback?
> > > > Best regards
> > > > David
> > > > 
> > > > 
> > > > -----Original Message-----
> > > > From: Christian Grothoff [mailto:address@hidden
> > > > Sent: 11 November 2010 16:12
> > > > To: David J Myers
> > > > Cc: 'libmicrohttpd development and user mailinglist'
> > > > Subject: Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd
> > > > 
> > > > Dear David,
> > > > 
> > > > You must use 'MHD_create_response_from_callback', not 'from_data';
> > > > use MHD_SIZE_UNKNOWN for the size argument and your data will be
> > > > transmitted while
> > > > you're still (incrementally) creating the response.
> > > > 
> > > > Happy hacking,
> > > > 
> > > > Christian
> > > > 
> > > > On Thursday, November 11, 2010 04:35:06 pm David J Myers wrote:
> > > > > Hi Christian,
> > > > > Thanks for the response. The infinite method is the one I need, but
> > > 
> > > could
> > > 
> > > > > you give me a bit more detail on the MHD calls I need to make.
> > > > > 
> > > > > I don't understand how to send each response part ie how to make an
> > > > > incremental response. In my callback, I can call
> > > > > MHD_Create_response_from_Data then MHD_queue_response, multiple
> > > > > times, but these responses won't be sent until the callback returns
> > > > > which won't work.
> > > > > 
> > > > > Thanks again,
> > > > > David
> > > > > 
> > > > > -----Original Message-----
> > > > > From: Christian Grothoff [mailto:address@hidden
> > > > > Sent: 11 November 2010 15:04
> > > > > To: libmicrohttpd development and user mailinglist
> > > > > Cc: David J Myers
> > > > > Subject: Re: [libmicrohttpd] JPEG Streaming with libmicrohttpd
> > > > > 
> > > > > Dear David,
> > > > > 
> > > > > MHD does very little for you here.  You'll need to set the content
> > 
> > type
> > 
> > > > > header
> > > > > (MHD_add_response_header) and then somehow create the response with
> > 
> > the
> > 
> > > > > boundaries yourself.  Depending on the requirements of your
> > > > > application, you
> > > > > 
> > > > > can either create it in-memory a-priori (if the stream is fixed
> > > > > size and small) or incrementally with the possibility of making
> > > > > the
> 
> stream
> 
> > > > > infinite in
> > > > > size (choose there respective MHD method to create the response
> > > > > object).
> > > > > 
> > > > > Either case will work, and in either case MHD will not help with
> > > > > the formatting of the response.
> > > > > 
> > > > > I hope this helps a bit.
> > > > > 
> > > > > Best,
> > > > > 
> > > > > Christian
> > > > > 
> > > > > On Thursday, November 11, 2010 01:30:52 pm David J Myers wrote:
> > > > > > Hi, I'm new to libmicrohttpd and I want to implement a motion
> > > > > > JPEG stream. Could someone please show me how to build the
> > > > > > following responses ie. A multipart response stream. This is
> > > > > > what I want to send back to the client when I receive the GET :-
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > HTTP Code: 200 OK
> > > > > > Content-Type: multipart/x-mixed-replace; boundary=myboundary
> > > > > > 
> > > > > > --myboundary
> > > > > > Content-Type: image/jpeg
> > > > > > 
> > > > > > <JPEG Image data>
> > > > > > --myboundary
> > > > > > Content-Type: image/jpeg
> > > > > > 
> > > > > > <JPEG Image data>
> > > > > > --myboundary
> > > > > > .
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > Thanks for your time
> > > > > > 
> > > > > > David
> > > > > 
> > > > > No virus found in this incoming message.
> > > > > Checked by AVG - www.avg.com
> > > > > Version: 9.0.869 / Virus Database: 271.1.1/3248 - Release Date:
> > > > > 11/10/10 19:34:00
> > > > 
> > > > No virus found in this incoming message.
> > > > Checked by AVG - www.avg.com
> > > > Version: 9.0.869 / Virus Database: 271.1.1/3248 - Release Date:
> > > > 11/10/10 19:34:00
> > > 
> > > No virus found in this incoming message.
> > > Checked by AVG - www.avg.com
> > > Version: 9.0.869 / Virus Database: 271.1.1/3248 - Release Date:
> > > 11/10/10 19:34:00
> > > 
> >   _____
> > 
> > No virus found in this message.
> > Checked by AVG - www.avg.com
> > Version: 10.0.1153 / Virus Database: 424/3250 - Release Date: 11/11/10