Re: lynx-dev http referer problem

[David Woolley]

> It's also possible that one day one of the "major" browsers will start
> suppressing this header (maybe optionally), for the same reasons as Lynx.

I have a feeling that this is already implemented in one of the big 2,
although it may have been dropped, under pressure from content providers.

> If you need to have session semantics, using cookies would be a better
> way to implement this - cookies were invented for this purpose.

1) In most cases where session semantics are required, rather than 
click trail data mining, you can embed the session ID in the URLs - the
one argument against this is where you want to track a session across
static pages, in which case it breaks cacheability.  Embedding the
session ID works on all browsers.  Cookies are a privacy issue, because
most cookies you receive are not used to form sessions, but to 
correlate your accesses between sessions, for market research
purposes.  (Non-persistent cookies are normally used only for sessions.)

2) The main reasons for insisting on Referer are not normally sessions
but:

- security by obscurity (you don't like the cosmetics of the proper
  HTTP authorisation mechanism, so you implement a login form that 
  redirects you to an unpublished URL and check that the Referer for
  all internal pages is one of your own);

- deep linking protection (you don't believe in the concept of the world
  wide web, but rather the Front Page concept of a web, where you have
  one or more self contained "webs" on your site, which may only be
  entered at their respective home pages.  Any access, other than through
  the home page, with an off site referer is rejected - this also
  breaks cacheability).

Some sites may insist on them because the site is more interested in 
the click trails than serving the user.

Note that some of the commonly available web server log analysis programs
make a feature of using form URL keywords from Referer to provide reports
on which AltaVista, etc., keywords were used to find the site.  This, of
course, is one of the invasion of privacy issues.