Re: [Lynx-dev] Re: failed to visit google

[Serge Munhoven]

Hi,

Inspiring from your trace, I hand-crafted requests and submitted them
with netcat6 [1] to host 64.233.167.104 on port 80.  My diagnose would be
that there is a very weird buffer-overflow in Google's server with
regard to "Accept:" headers.  Depending on the number of "Accept:" headers
and their size, the request fails (400) or succeeds (302 in my case:
redicrection to google.be). E.g., the following:

GET / HTTP/1.0
Host: www.google.com
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: text/html
Accept: 
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx


is OK.  But if you add a character on any line or append another "Accept:" line
(even an empty one!), you'll get a 400-Bad-request.  The actual contents
does not seem to matter.  For instance, replacing some or all of the x's
by "text/html, text/html, ..." yields the same result.
I've got variants with more "text/html"'s per line, requiring more or less long
last line.  E.g:

GET / HTTP/1.0
Host: www.google.com
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: text/html, text/html, text/html, text/html
Accept: xxxxxx


I dont understand exactly what overflows when. (Cf. a similar problem discussed
in August 2002 on this list: e.g, http://www.sysinternals.com server returned
a 406 when the "Accept:"-headers totalized more than 256 characters - but this
was an IIS; isn't google supposed to use Linux ? :).


Conclusion: I don't really see how lynx should handle this differently.
Maybe check how other browsers handle a lot of mimetypes ?

Possible workaround: simplify your ~/.mailcap (e.g. merge subtypes into
"image/*", "video/*", "audio/*").  Anyway, you could try to temporarily
rename your ~/.mailcap (and maybe its system-wide equivalent) and try to
access goggle to confirm that this is indeed the issue.

Regards,

 - Serge

[1] http://netcat6.sourceforge.net/

On Fri, May 07, 2004 at 10:33:24AM -0400, Larry W. Virden wrote:
[...]
 >  HTParseInet: Parsed address as port 80, IP address 64.233.167.104
[...]
 >  Writing:
 >  GET / HTTP/1.0\r
 >  Host: www.google.com\r
 >  Accept: text/html, text/plain, application/vnd.sun.xml.writer, 
 > application/vnd.sun.xml.writer.global, application/vnd.stardivision.writer, 
 > application/vnd.stardivision.writer-global, application/x-starwriter, 
 > application/vnd.sun.xml.writer.template\r
 >  Accept: application/vnd.sun.xml.calc, application/vnd.stardivision.calc, 
 > application/x-starcalc, application/vnd.sun.xml.calc.template, 
 > application/vnd.sun.xml.impress, application/vnd.stardivision.impress, 
 > application/vnd.stardivision.impress-packed\r
 >  Accept: application/x-starimpress, 
 > application/vnd.sun.xml.impress.template, application/vnd.sun.xml.draw, 
 > application/vnd.stardivision.draw, application/x-stardraw, 
 > application/vnd.sun.xml.draw.template, application/vnd.sun.xml.math\r
 >  Accept: application/vnd.stardivision.math, application/x-starmath, 
 > audio/x-mpegurl, image/jpeg, image/gif, image/x-xwd, image/postscript, 
 > image/ps, image/*, message/partial, message/external-body, 
 > application/x-dvi, application/postscript, x-be2\r
 >  Accept: application/andrew-inset, text/richtext, text/enriched, 
 > application/framemaker, application/view_pctdoc, application/x-tgif, 
 > multipart/enabled-mail, application/safe-tcl, application/x-ccitcl, 
 > application/pgp, application/applefile\r
 >  Accept: text/tab-separated-values, application/x-chat, application/x-xdma, 
 > application/x-rasmol, application/x-wt, video/x-mpeg, x-world/x-vrml, 
 > application/x-candleweb, x-conference/x-cooltalk, audio/midi, 
 > application/x-bzip2, application/marimba\r
 >  Accept: application/pdf, video/quicktime, video/x-ms-asf, video/x-ms-wmv, 
 > video/x-ms-wvx, audio/x-ms-wax, audio/x-ms-wma, video/x-ms-asf-plugin, 
 > application/x-mplayer2, video/x-ms-wm, video/x-ms-wmx, video/x-ms-wmp, 
 > application/x-java-jnlp-file\r
 >  Accept: video/mpeg, video/x-msvideo, video/x-mpeg2, audio/x-pn-realaudio, 
 > audio/vnd.rn-realaudio, application/smil, text/vnd.rn-realtext, 
 > video/vnd.rn-realvideo, application/x-shockwave-flash2-preview, 
 > application/sdp, application/x-sdp\r
 >  Accept: application/vnd.rn-realmedia, audio/wav, audio/x-wav, 
 > audio/x-pn-wav, audio/x-pn-windows-acm, audio/basic, audio/x-pn-au, 
 > audio/aiff, audio/x-aiff, audio/x-pn-aiff, text/sgml, */*;q=0.01\r
 >  Accept-Encoding: gzip, compress\r
 >  Accept-Language: en\r
 >  User-Agent: Lynx/2.8.5dev.16 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6\r
 >  From: address@hidden
 >  Cookie2: $Version="1"\r
 >  Cookie: 
 > PREF=ID=6ecf369a3d4ea08b:TM=1011483183:LM=1033390392:S=EHm4COeBmlVOcgsQ\r
 >  \r