bug#35969: 26.2, Excorporate

[tenspd137 .]

I haven't tried the first item on the list yet - ie reloading Andreas'
function.  i have tried to build emacs from
https://github.com/emacs-mirror/emacs.git because I can't reach the
official repo from my firewall for whatever reason.  I am sure it is
something here.  Anyway, I was having problems because my built
version (27.0.x) was getting seemingly mixed up with my current
installation.  I can try again later, but right now things are pretty
busy at work.

Thanks.

On Tue, Jun 18, 2019 at 10:26 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>
> Hi,
>
> "tenspd137 ." <dcday137@gmail.com> writes:
>
> > So - I am not sure if I did it correctly, but I copied this function
> > with Andreas' changes into a file:
> >
> > (defun url-https-proxy-after-change-function (_st _nd _length)
> >   (let* ((process-buffer (current-buffer))
> >          (proc (get-buffer-process process-buffer)))
> >     (goto-char (point-min))
> >     (when (re-search-forward "^\r?\n" nil t)
> >       (backward-char 1)
> >       ;; Saw the end of the headers
> >       (setq url-http-end-of-headers (set-marker (make-marker) (point)))
> >       (url-http-parse-response)
> >       (cond
> >        ((null url-http-response-status)
> >         ;; We got back a headerless malformed response from the
> >         ;; server.
> >         (url-http-activate-callback)
> >         (error "Malformed response from proxy, fail!"))
> >        ((= url-http-response-status 200)
> >         (if (gnutls-available-p)
> >             (condition-case e
> >                 (let ((tls-connection (gnutls-negotiate
> >                                        :process proc
> >                                        :hostname (url-host 
> > url-current-object)
> >                                        :verify-error nil)))
> >                   ;; check certificate validity
> >                   (setq tls-connection
> >                         (nsm-verify-connection tls-connection
> >                                                (url-host url-current-object)
> >                                                (url-port 
> > url-current-object)))
> >                   (with-current-buffer process-buffer (erase-buffer))
> >                   (set-process-buffer tls-connection process-buffer)
> >                   (setq url-http-after-change-function
> >                         'url-http-wait-for-headers-change-function)
> >                   (set-process-filter tls-connection 
> > 'url-http-generic-filter)
> >                   (process-send-string tls-connection
> >                                        ;; Use the non-proxy form of the 
> > request
> >                                        (let (url-http-proxy)
> >                                          (url-http-create-request))))
> >               (gnutls-error
> >                (url-http-activate-callback)
> >                (error "gnutls-error: %s" e))
> >               (error
> >                (url-http-activate-callback)
> >                (error "error: %s" e)))
> >           (error "error: gnutls support needed!")))
> >        (t
> >         (url-http-debug "error response: %d" url-http-response-status)
> >         (url-http-activate-callback))))))
> >
> > and then loaded it before running excorporate.  After that, I did M-x
> > excorporate, and the minibuffer returns:  error in process filter:
> > Server response is not an XML document
>
> In this scenario, if you immediately (without restarting Emacs/reloading
> anything) re-run M-x excorporate does it still fail?  I just want to
> make sure that's not a transient failure.  If it does fail the second
> time, can you post the HTTP response from the server?
>
> > When I do the similar test by loading the url-http-create-request with
> > Thomas's changes, I can get a connection and grab my schedule
> > through the proxy.
>
> OK.
>
> > Let me know if I need to try something different.
>
> Are you in a position to build Emacs master tip and retry the experiment
> without patching anything?
>
> Thanks,
> Thomas
>
> > On Mon, Jun 17, 2019 at 4:08 PM Thomas Fitzsimmons <fitzsim@fitzsim.org> 
> > wrote:
> >>
> >> Hi,
> >>
> >> Good to hear that the patch I posted worked!
> >>
> >> Yes, that's the patch that Andreas's commit
> >> 84613dae5c34ea742dd9a3e56f5acb55f604b483 applied.  From what I can tell,
> >> you will not have that in Emacs 26.2.
> >>
> >> Can you try reverting my patch and applying Andreas's patch, and see if
> >> Excorporate still works through the proxy?
> >>
> >> Thanks,
> >> Thomas
> >>
> >> "tenspd137 ." <dcday137@gmail.com> writes:
> >>
> >> > The patch Thomas seems to work from behind the proxy.  My current
> >> > emacs version is 26.2, so I would think it would include the commit
> >> > Andreas is talking about....  I went and looked it up - is this the
> >> > correct commit?
> >> >
> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
> >> > index 53798f7..817c5ce 100644
> >> > --- a/lisp/url/url-http.el
> >> > +++ b/lisp/url/url-http.el
> >> > @@ -1412,7 +1412,9 @@ The return value of this function is the
> >> > retrieval buffer."
> >> > 'url-http-wait-for-headers-change-function)
> >> > (set-process-filter tls-connection 'url-http-generic-filter)
> >> > (process-send-string tls-connection
> >> > - (url-http-create-request)))
> >> > + ;; Use the non-proxy form of the request
> >> > + (let (url-http-proxy)
> >> > + (url-http-create-request))))
> >> > (gnutls-error
> >> > (url-http-activate-callback)
> >> > (error "gnutls-error: %s" e))
> >> >
> >> > Thanks!
> >> >
> >> > -C
> >> >
> >> >
> >> > On Sat, Jun 15, 2019 at 1:41 AM Andreas Schwab <schwab@linux-m68k.org> 
> >> > wrote:
> >> >>
> >> >> On Jun 14 2019, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
> >> >>
> >> >> > diff --git a/lisp/url/url-http.el b/lisp/url/url-http.el
> >> >> > index 00803a103a..723d111d58 100644
> >> >> > --- a/lisp/url/url-http.el
> >> >> > +++ b/lisp/url/url-http.el
> >> >> > @@ -329,7 +329,10 @@ url-http-create-request
> >> >> >               ;; The request
> >> >> >               (or url-http-method "GET") " "
> >> >> >               (url-http--encode-string
> >> >> > -              (if using-proxy (url-recreate-url url-http-target-url) 
> >> >> > real-fname))
> >> >> > +              (if (and using-proxy
> >> >> > +                       (not (equal "https" (url-type 
> >> >> > url-http-target-url))))
> >> >> > +                  (url-recreate-url url-http-target-url)
> >> >> > +                real-fname))
> >> >>
> >> >> That should already be handled by commit 84613dae5c.
> >> >>
> >> >> Andreas.
> >> >>
> >> >> --
> >> >> Andreas Schwab, schwab@linux-m68k.org
> >> >> GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
> >> >> "And now for something completely different."