Re: [Lynx-dev] sudden site access failure: realclearpolitics.com

[Rudy Vener]

Hi Karen,

Here's the URL from my rcp script:

http://www.realclearpolitics.com

On Wed, Nov 10, 2021 at 04:15:21PM -0500, Karen Lewellen wrote:
> Hi Rudy,
> would you mind placing here the exact link you are using?
> And, yes at least in some places, cloudflare blocks links the chain too.
> If you place what you are using, I will do my text again, sharing what I am
> getting.
> Would that resonate?
> Kare
> 
> 
> 
> On Wed, 10 Nov 2021, Rudy Vener wrote:
> 
> > Hi Karen,
> > They are probably not blocking by IP or they would have blocked access from 
> > links (the chain) as well.
> > 
> > Thinking they might be blocking by a combination of both IP and User agent 
> > trigger, I changed the user agent
> > for lynx (the cat) with another UA string I got from the online UA database.
> > Now I called  realclearpolitics.com with:
> > /usr/local/bin/lynx-2.9.0-dev9 -useragent="Mozilla/5.0 (Macintosh; U; PPC 
> > Mac OS X Mach-O; XH; rv:8.578.498) fr, Gecko/20121021 Camino/8.723+ 
> > (Firefox compatible)" $*
> > 
> > No joy. Still blocked with lynx (the cat.)  Links (the chain) still works.
> > 
> > I don't suppose you could try sending me shellworld's lynx config file, 
> > could you?
> > 
> > On Wed, Nov 10, 2021 at 02:06:45PM -0500, Karen Lewellen wrote:
> > > Hi Rudy,
> > > Have  you tried this site today?
> > > Granted I am using the Lynx configuration at shellworld, but I just typed
> > > the  address you provided, and reached the page perfectly.
> > > something to consider about cloudflare is that it tracks sometimes by IP
> > > address.
> > > Would love to learn how the fanfiction.net site works with your user agent
> > > though, smiles.
> > > Karen
> > > 
> > > 
> > > 
> > > On Wed, 10 Nov 2021, Rudy Vener wrote:
> > > 
> > > > Hi all,
> > > > 
> > > > Yesterday realclearpolitics.com started returning the following page
> > > > when I tried to access it with lynx.
> > > > 
> > > >   Please enable JS and disable any ad blocker
> > > > 
> > > > I invoke lynx as follows:
> > > > /usr/local/bin/lynx-2.9.0-dev9 -useragent="Mozilla/5.0 (Macintosh; 
> > > > Intel Mac OS X 10_8_0) AppleWebKit/537.1 (KHTML, like Gecko) 
> > > > Chrome/21.0.1180.79 Safari/537.1 lynx" $*
> > > > 
> > > > The above user agent spoof normally lets me slip the Cloud Flare 
> > > > gatekeepers, but this seems to be something new.
> > > > 
> > > > For what it's worth, the same problem now occurs with w3m, however,
> > > > links (the chain)  can access the site just fine.
> > > > If there is a simple configuration fix for this, I'd really like to 
> > > > know.
> > > > 
> > > > Thank you.
> > > > 
> > > > --
> > > > Rudy Vener
> > > > Website: http://www.rudyvener.com
> > > > Twitter: https://twitter.com/RudySalt
> > > > 
> > > > 
> > 
> > -- 
> > Rudy Vener
> > Website: http://www.rudyvener.com
> > Twitter: https://twitter.com/RudySalt
> > 
> > 

-- 
Rudy Vener
Website: http://www.rudyvener.com
Twitter: https://twitter.com/RudySalt