bug#27658: 26.0.50; response parsing hangs when using tls.el and gnutls-cli.exe on Windows

[Richard Copley]

Install GNUTLS. Obtain certificates and set `tls-program' as described
in Step 3 of Glyph's blog post at
"https://glyph.twistedmatrix.com/2015/11/editor-malware.html";.

Ensure Emacs is using "gnutls-cli.exe" instead of "libgnutls-30.dll"
by (a) removing the DLL from your path or (b) configuring Emacs with
the "--without-gnutls" configure option.

In emacs -Q:

(pop-to-buffer
 (url-retrieve
  "https://elpa.gnu.org/packages/archive-contents";
  #'ignore))

Expected result: a buffer is displayed and, soon afterwards, filled with
the HTTP response (with the network CRLF line-endings in the raw
response translated to ordinary newlines, without CR, in the buffer).

Actual result: "Opening TLS connection [...]...done" is shown in
the minibuffer and the function never returns. (Type C-g to quit).

(This is probably the cause of #22929.)

----
Observing the process filter in the debugger, there is an extra carriage
return before every newline in the process buffer of the gnutls-cli.exe
process.

By "extra", I mean the informational text from "gnutls-cli.exe" has
one carriage return at the end of each line (where there should be none)
and the HTTP headers have two carriage returns at the end of each line
(where there should be one, because this is the raw payload and has
network line endings).

The extra newlines in the informational output cause `open-tls-stream'
in "tls.el" to hang indefinitely because the regexp `tls-end-of-info'
never matches. Fixing that regexp is not a solution. When I try that,
the HTTP header parsing in "url-http.el" fails in a similar way.

Possible fix:

The other obvious experiment I tried was to add
"(set-process-coding-system process 'raw-text-dos)" to
`open-tls-stream', just after the call to `start-process'. (Line
238.) Having done that, I do get a reasonable HTML document (and no
error) from `url-retrieve'.

That's a data point for information. I'm not sure if it's a good fix.
In the blog post I referred to above, Glyph says:

"[...] for some reason, Emacs could not parse gnutls-cli.exe   s output no
matter what I did."

Indeed, with the change(s) above, "package.el" still doesn't
succeed in parsing package lists retrieved over HTTPS; but that
might be a separate bug.
----

In GNU Emacs 26.0.50 (build 1, x86_64-w64-mingw32)
 of 2017-07-11 built on MACHINE
Repository revision: 7168bd2957a567c235d1b8b57acb4325187bd542
Windowing system distributor 'Microsoft Corp.', version 10.0.15063
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --prefix=/mingw64 --config-cache --with-modules
 --without-gnutls --without-pop --enable-locallisppath=/c/emacs-lisp
 CFLAGS=-O3'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS MODULES

Important settings:
  value of $EMACSLOADPATH: c:\emacs-lisp;
  value of $LANG: ENG
  locale-coding-system: cp1252

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message subr-x puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib dired dired-loaddefs
format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg
epg-config gnus-util rmail rmail-loaddefs mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date mule-util
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote w32notify w32 multi-tty make-network-process emacs)

Memory information:
((conses 16 96670 10234)
 (symbols 56 20064 1)
 (miscs 48 39 87)
 (strings 32 29950 1608)
 (string-bytes 1 773498)
 (vectors 16 13875)
 (vector-slots 8 480519 8489)
 (floats 8 50 151)
 (intervals 56 240 3)
 (buffers 976 11))