On Sunday 14 November 2010 22:09:35 Jared Cantwell wrote:
> I agree that there should be a cleaner way to handle this. Our goal is
> simply to provide a URL through which content can be downloaded, and we
> would like to not have to specify the client used (nor write our own).
>
> I can see many reasons why a chunked response can fail. For example, data
> corruption or failed disks, both of which may only be detected once the
> response has started. My understanding of the purpose behind chunked
> encoding in the first place is to not have to completely prepare the
> response ahead of time. However, what if after partially preparing the
> response, an unrecoverable error occurs? Based on my readings of the RFC,
> there doesn't seem to be a standard way of handling this situation.
>
> Given the reasonable possibility of this situation, does it make sense for
> the API to provide a way to "abort" the response? I think that closing the
> connection is a nice way to signal the other end that a fatal error has
> occurred, don't you? To handle it otherwise, the developer would need to
> control both the client and the server, which seems unnecessary to me.
>
> I can understand if you don't wish to incorporate this into your API, but
> would you mind giving a few pointers on where the code would need modified
> to make this happen (as a test run)?
>
> Just a thought
>
> Jared
>
> On Sun, Nov 14, 2010 at 3:47 PM, Christian Grothoff
>
> <
address@hidden>wrote:
> > Well, this is not something a well-behaved HTTP server should ever do, so
> > I think the solution with the footers is cleaner (and we should probably
> > limit
> > the API to "legal" HTTP behavior --- after all, if the response
> > construction
> > could realistically fail, the developer should take care of this case in
> > a nice way).
> >
> > My 2 cents
> >
> > Christian
> >
> > On Sunday, November 14, 2010 09:07:08 pm Jared Cantwell wrote:
> > > Thanks Christian!
> > >
> > > I think this will help a lot. Would it make sense to have the ability
> > > to prematurely close the connection with some error code (i.e. by
> > > returning -2)? This is similar to what the try_ready_normal_body()
> > > does. I would imagine that any normal client would interpret this as
> > > a failed request since they would have never received the trailing
> > > '0\n\r'.
> > >
> > > ~Jared
> > >
> > > On Sun, Nov 14, 2010 at 2:49 PM, Christian Grothoff
> > >
> > > <
address@hidden>wrote:
> > > > Dear Jared,
> > > >
> > > > I was just about to fire back a response saying: "just add a footer",
> >
> > but
> >
> > > > then
> > > > I decided to look for the API for that and I found that -- while 99%
> > > > of the support for this is there -- a tiny function to do so was
> > > > missing from the API.
> > > >
> > > > So I'm happy to introduce: "MHD_add_response_footer"; the only
> >
> > difference
> >
> > > > to
> > > > 'MHD_add_response_header' being that the 'kind' is set to
> > > > MHD_FOOTER_KIND. The
> > > > new API is in SVN 13683.
> > > >
> > > > This helps you in that you simply add a footer that signals your
> > > > client software that some kind of error happened; doing so inside of
> > > > the response callback while also returning "-1" should do the trick.
> > > >
> > > > In fact, the recommended approach would actually be to always add a
> > > > cryptographic checksum (MD5, SHA-1) as a footer. That way, your
> >
> > clients
> >
> > > > will
> > > > also be better at detecting other transmission errors in your
> > > > gigabytes of data. This is the best approach, and certainly also
> > > > how HTTP intended footers
> > > > to be used.
> > > >
> > > > Naturally, this only works if you can teach your HTTP client to
> > > > respect footers. If you cannot do that, I don't have a good answer
> > > > for you.
> > > >
> > > > Happy hacking,
> > > >
> > > > Christian
> > > >
> > > > On Sunday, November 14, 2010 06:00:37 pm Jared Cantwell wrote:
> > > > > Hi Christian,
> > > > >
> > > > > Thanks for a great piece of software-- it has helped me
> > > > > tremendously
> >
> > to
> >
> > > > > quickly implement a key feature for us. I have a question about
> > > > > chunked uploads that I was hoping you could help me with.
> > > > > According to the documentation (and my reading of the code), when
> > > > > using MHD_ContentReaderCallback to return chunks of data (i.e. in
> > > > > response
> >
> > to
> >
> > > > > a GET request), there doesn't seem to be a way to indicate that the
> > > > > chunked response failed. Returning 0 will try again, -1 will
> > > > > assume the transfer completed successfully. Is there a way to
> > > > > indicate to MHD that a chunked response failed? The reason I ask
> > > > > is that we are transferring files on
> > > >
> > > > the
> > > >
> > > > > order of 10s of GBs, and we're getting this data from throughout
> > > > > our cluster. Midway through the transfer, anything could happen,
> > > > > and it
> > > >
> > > > would
> > > >
> > > > > be nice to just prematurely close the connection to signal failure
> > > >
> > > > (instead
> > > >
> > > > > of assuming the end of the chunked data was reached). Am I missing
> > > > > something in the interface, or is this just not currently possible?
> > > > >
> > > > > Thanks for any help!!
> > > > > ~Jared