Re: [Lynx-dev] Lynx fails on http://politiken.dk

[David Combs]

On Mon, Oct 11, 2004 at 04:21:22PM +0000, Thorsten Glaser wrote:
> Morten Bo Johansen dixit:
> 
> >The URL above redirects to http://politiken.dk/VisArtikel.iasp?PageID=1
> >but the server replies 404. Typing in the redirect URL directly produces
> >the same result.
> 
>    Internet Information Services
> 
> Sure it's lynx? :-)
> 
> 
> /me tcpdumps
> 
> Judge yourself:
> 
> => Lynx
> GET /VisArtikel.iasp?PageID=1 HTTP/1.0
> Host: politiken.dk
> Accept: text/html, text/plain, text/sgml, */*;q=0.01
> Accept-Language: en
> Accept-Charset: utf-8, iso-8859-1;q=0.01, us-ascii;q=0.01
> User-Agent: Mozilla/5.0 (UNIX; U; OpenBSD; C; compatible) 
> Lynx/2.8.6dev.5-MirOS libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7d 
> Revision/MirOS:BSD (https://MirBSD.BSDadvocacy.org:8890/) fake (MSIE 5.5)
> Cookie2: $Version="1"
> Cookie: ASPSESSIONIDSQRBRARD=NEGNMMOAIEBAMINAINIBIFGJ
>       => (404)
> 
> => Links+
> GET /VisArtikel.iasp?PageID=1 HTTP/1.1
> Host: politiken.dk
> User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
> Accept: */*
> Accept-Charset: us-ascii, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, 
> ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, 
> ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, windows-1250, 
> windows-1251, windows-1252, windows-1256, windows-1257, cp437, cp737, cp850, 
> cp852, cp866, x-cp866-u, x-mac, x-mac-ce, x-kam-cs, koi8-r,koi8-u, koi8-ru, 
> TCVN-5712, VISCII, utf-8
> Connection: Keep-Alive
> Cookie: ASPSESSIONIDSQRBRARD=AIGNMMOAHNDPFBBOLMHAAOCG
>       => (works)
> 
> => Netcat
> GET /VisArtikel.iasp?PageID=1 HTTP/1.0
> Host: politiken.dk
> Accept: text/html, text/plain, text/sgml, */*;q=0.01
> Accept-Language: en
> Accept-Charset: utf-8, iso-8859-1;q=0.01, us-ascii;q=0.01
> User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
> Cookie2: $Version="1"
> Cookie: ASPSESSIONIDSQRBRARD=NEGNMMOAIEBAMINAINIBIFGJ
>       => (works)
> 
> 
> They are actively blocking anyone with libwww-FM/2.14 in their user agent
> string (I tested for that). (Actually, libwww is enough.)
> 
> That's not a bug in lynx, and you can change the user agent in the
> (O)ptions page.
> On the other hand, I would consider contacting them and asking if
> they are still all in their right mind...
> 
> bye,
> //mirabile
> 
> 
> _______________________________________________
> Lynx-dev mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/lynx-dev



Naive questions:

(1) What in the above trace tells us that they
    are blocking libwww in user-agent strings?

    I do see "accept-" this or that, but no "reject"
    or the like.

   Obviously, I'm missing something that's obvious
   to everyhone else.  Just what is i


(2) where did the libwww in his UA-string come *from*?

    I mean, did he have it there explicitly in his .lynxrc
    or lynx.cfg, or perhaps typed-in by hand via the option page?

    Or did it "expand" from some variable or macro-like-thing in
    his string. 

    Or was it added (appended?) by his site, somehow?


Thanks!

David