Re: lynx-dev Location: redirects and reading data

[Klaus Weide]

On Sun, 28 May 2000, brian j pardy wrote:

> On Sun, May 28, 2000, David Woolley wrote:
> > > Does anybody have any reason we shouldn't close the connection after
> > > we've received all of the server's headers and there is a Location:
> > > redirect and we're using a GET or HEAD?  (Other than it being rude, of
> > > course.)
> > 
> > It breaks persistent connections, and would be triggered on every
> > redirect, as all are supposed to include some fallback text.  Even 
> > if Lynx doesn't use persistent connections, a proxy might.  Lynx probably
> > should use them, although, without frames or images, most will time out
> > except when crawling.
> 
> Is there a typical timeout for persistent connections specified in the
> spec somewhere (maybe just a TCP stack timeout)?  Or is it something left
> up to individual server operators?  If there's anything specifying a
> timeout of more than 30-45 seconds or so it might be worth doing -- I
> can often find the link I want to go to that quickly.  I assume the
> work coding wouldn't be worth it, though, for the minimal gain.

Lynx doesn't do persistent HTTP connections.  (Not even a HTTP 1.1 is
required to do them.)  Adding support would be an interesting (and
complex) project, but for a client that doesn't request inline images
etc., the benefit would be minimal.

Both client and server can break a (HTTP 1.1) persistent connection
anytime they like, and the other end has to be prepared to deal with
that at any point.  There isn't anything in the specs suggesting a
value, as far as i am aware.  But AFAIK (anyone with better
information?), servers typically use short timeouts - long enough for
one page and its inline images and objects to be handled over one
connection (or a set of parallel connections), but too short to be
likely still around when the next link is followed (assuming the user
actually reads something on the page).

You can survey server behavior for yourself:

(lines typed in, including final Enter, shown prefixed with '>>')

----------------------------------------------------------------
$ >>telnet www.apache.org 80
Trying 63.211.145.10...
Connected to www.apache.org.
Escape character is '^]'.
>>HEAD / HTTP/1.1
>>Host: www.apache.org
>>
HTTP/1.1 200 OK
Date: Sun, 28 May 2000 23:36:53 GMT
Server: Apache/1.3.9 (Unix) ApacheJServ/1.1 PHP/3.0.12 AuthMySQL/2.20
Cache-Control: max-age=86400
Expires: Mon, 29 May 2000 23:36:53 GMT
Content-Type: text/html

Connection closed by foreign host.
----------------------------------------------------------------

Note length of pause before "Connection closed" appears.
Note the "HTTP/1.1".  For a "HTTP/1.0" request, the connection
is closed immediately.

> As an aside -- do we *ever* do anything with that fallback text?

It gets blackholed.  But you may see part of it in the trace log.

When you use -noredir it gets shown of course.  (And the redirection
is not followed.)

> Store it somewhere, perhaps to show if we get an error from the location
> specified in the header?

The way things are done now, that would be much to late.  Bytes are
discarded on receipt (after the header has been parsed).

Note that the previous implementation (up to 2.8.2) was less efficient in
that respect.  It would first receive the full message into a buffer
and then search for headers in it.  Now things are happeneing more
on the fly.

> Should we?

I don't think it's useful enough to bother.  You can use trace or -noredir
or ']' (HEAD) if you really want to know something about the redirection
message itself.

   Klaus


; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden