bug#35658: 27.0.50; Problems with chunked/gzipped content in url-http

[Lars Ingebrigtsen]

To reproduce, say:

(url-retrieve-synchronously 
"https://www.facebook.com/verkstedetbar/events/?ref=page_internal";)

This now appears to spin until Facebook closes the connection on their
side.

Looking at the data, it appears that Facebook outputs chunked gzipped
data, which is pretty normal.  What's not normal is that what's in the
http buffer is

HTTP/1.1 200 OK
Content-Encoding: gzip
[...]
Date: Thu, 09 May 2019 17:45:53 GMT
Transfer-Encoding: chunked
Connection: keep-alive

3^_\213^H^@^@^@^@^@^@...

(I've transliterated the binary data into what it looks like in the
buffer.)

I.e., there's not chunking header before that gzip file start.  Somehow,
either Emacs is removing the leading

4af^M

and replacing it with "3", or Facebook is outputting something odd.  I
tend to believe it's the former, since curl (and other programs)
understand the chunked data perfectly.

It's most strange...

If we switch gzip http off by saying

(setq url-mime-encoding-string nil)

then the chunked headers are correct and everything works.



In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2019-05-02 built on sandy
Repository revision: d4fa998c3142b5ae13664295dcf2136397b05f5a
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 18.04.2 LTS


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no