Re: lynx-dev Why doesn't lynx cache HTML source?

[David Woolley]

> >Pages are becoming uncacheable either because they are dynamically
> >generated or because uncacheability is forced in order to give the
> 
> Why does dynamic generation make a page uncacheable?

This was written in the context of someone talking about following the
formal HTTP caching rules.  Dynamic pages can be made cacheable but that
requires a positive effort by their authors and is often contrary to the
(often commercial) reasons why they were made uncacheable.

For a page to be cacheable by current generation caches it needs to have
either explicit expiry date information or a Last-Modified header.  Some
caches may insist on it having a length as well.  Dynamically generated
pages normally have neither although it is possible to set them.  In this
case, the cache needs to reverify the page every time, but there is 
insufficient data to allow this to be done on a typical dynamic page,
and it is a lot of extra work for the programmer, so most dynamic 
pages will just return a fresh copy in response to an If-Modified-Since
request.

Current generation GUI clients do not do what the article I was replying
to suggested, but only reverify once per session.  The article that I was
responding to appeared to suggest that Lynx must not use this policy, but
must use an always revalidate policy and it was in that context that I
pointed out the problem of dynamic pages.

The particular problem with AltaVista banner adverts is that the URL used
includes the search keywords, so varies every time.  It's also in the form
of a form reference and previous abuse of GET mode forms for really 
uncacheable material has meant that the HTTP 1.1 RFC and the default 
configuration for Squid treat a ? as indicating uncacheable material.
Obviously Lynx doesn't have a serious problem with GIF in banner adverts!

Incidentally, I believe that IMDB do make their dynamic pages cacheable,
with Expires or Cache-Control headers, but set a rather short time limit.
They have commercial reasons for wanting each real new request to generate
a new hit on the main site as they want to present continually changing
adverts.