bug#22493: 25.1.50; open-gnutls-stream doesn't respect :nowait, so the connections are synchronous

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 22493@debbugs.gnu.org
> Date: Sat, 30 Jan 2016 23:55:57 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The problem is, though, that the process isn't marked as a gnutls
> >> process until gnutls_boot is called, so this needs to happen much
> >> earlier.  Possibly by adding another keyword to make-network-process
> >> that just sets
> >> 
> >>   XPROCESS (proc)->gnutls_p = 1;
> >
> > Why do you need to mark a process gnutls_p, when it definitely still
> > isn't, not until the GnuTLS negotiation is successfully completed?
> 
> It is a socket that will only be used for TLS.  And the current way
> we're creating TLS streams is by calling `open-gnutls-stream', which
> (simplified) just calls `make-network-stream' and then `gnutls-boot' in
> rapid succession.  So any stream you get out of `open-gnutls-stream'
> will look just the same -- it'll have gnutls_p set.
> 
> It'll just happen slightly earlier.

There's a certain semantics to this flag being set, and overloading it
doesn't sound like a good idea to me.

But this is a minor issue -- the larger one is below.

> >> #ifdef HAVE_GNUTLS
> >>          if (p->gnutls_p && p->gnutls_state)
> >>            written = emacs_gnutls_write (p, cur_buf, cur_len);
> >>          else
> >> #endif
> >>            written = emacs_write_sig (outfd, cur_buf, cur_len);
> >> 
> >> I think that looks rather nonsensical.
> >
> > Does this happen today?  If so, in what scenario?
> 
> It does not happen today, but it will happen if we open the TLS stream
> :nowait.

So not doing that will avoid this problem.  Good.

> > If you want GnuTLS negotiation to happen from the sentinel, you need
> > some machinery to block such process objects from communicating.
> > I'm not sure silently skipping I/O is TRT for that: won't Lisp
> > programs that happen to start communicating too early become confused
> > about what's going on?
> 
> The "GET /" will block, sort of, as it would on any socket that's backed
> up.

That depends on what the (as yet unwritten) code will do when it gets
to the above fragment and discovers that GnuTLS did not yet finish
handshaking.  You didn't really describe what you intend to do.  Your
options are (a) silently do nothing, or (b) fail the write.  Either
way sounds problematic to me.

> > More generally, what advantages do we expect to gain by making these
> > changes?  Let DNS resolution proceed in the background, only to have
> > to wait for GnuTLS negotiations anyway, before we can start using the
> > stream?  Or are there more significant advantages?
> 
> It makes the entire connection, both the DNS resolution and the TLS
> negotiation, asynchronous.  So Emacs doesn't hang while eww is
> connecting to https://google.com.

I guess I'm missing something important here, because I don't
understand how can this work.  Async DNS works because getaddrinfo_a
starts threads under the hood to perform the resolution, while the
main (a.k.a. "Lisp") thread is free to do other things, until a call
to 'pselect' tells it the DNS resolution is complete.  By contrast,
the GnuTLS negotiation, whether it runs from a sentinel or from inside
open-network-stream, runs in the main thread, so Emacs must wait for
it to finish before it can do anything else.  Running it from a
sentinel doesn't magically make it asynchronous, since sentinels run
in the main thread, and the main thread is busy as long as the
sentinel runs.  Maybe you thought about running gnutls-boot from a
separate thread, but I see no easy way of doing that, without some
very thorough rewrite, since current code access Lisp data structures
an (at least optionally) displays logging messages in the echo area.

So how will this work asynchronously?  What am I missing?