lynx-dev "?" in URL fails from Perl script (was: Lynx Question)

[David Woolley]

All questions on this list are Lynx Questions, your Subject might as well
have been blank!!!

> 
> I'm trying to run lynx from within a perl script but get an error back
> when i try to return a url which includes the '?' character.

Using shell commands when you don't fully understand the handling of
various characters by the shell can seriously damaage your security.
In fact, there is no evidence here that you even attempted to open such
a URL.

> $SITELIST =
> "www.thisisstaffordshire.co.uk/scripts/edsection.asp?section=Local+sports";
> 
> open(SITE, $SITELIST);

Attempts to open a file whose name consists of
"edsection.asp?section=Local+sports" in the pre-existing scripts
sub-directory of the pre-existing www.thisisstaffordshire.co.uk
sub-directory of the current directory.

> while ($site = <SITE>) {

Reads each line from that file, so you are either living very dangerously,
or the file is not a previous dump of the URL the name implies.

> 
> open(FILE,"/opt/bin/lynx -dump $site |") or die "can't open header
> file";

Attempts to open a pipe with Lynx.  There is no shell meta
character protection, so `rm -Rf /*` in the first file will have the
(un-)expected disastrous side effect.

Note.  There is no evidence that there was a question mark in the value
of $site, although, if there had been the shell would try to expand it,
and if there was no match would either reject the command or let the
untransformed string through.

> 
> while ($line = <FILE>) {

Reads a line.

> 
>                 if ($line =~ /shrews/i) {
>                 print "$line \n";

Print it if it contains the name of your favourite team.

>                 }
>         }
> }
> 
> Lynx version is 2.7.1f

Was using a version of Lynx too old to be supported and unknown versions of
Perl and the shell.

Including the actual error messages would have confirmed whether the
first or second open was failing and exactly how.

Note.  If any of the pages contain advertising, or you do this with any
regularity, the content provider will probably not appreciate what you are
doing.  You may want to consider overriding the user agent to identify 
your tool, rather than Lynx, as the culprit, to prevent Lynx being 
barred from the site.
 
> I don't know if this is the right place to ask this question, if it
> isn't
> i'm sorry.

No it isn't, but it is an honest mistake this time.  However apologising
in advance makes it looks like the apology is a substitute for research;
if your research convinces you that you have the right group, go with
your convictions; if you still have doubts, explain why you chose the
group and rejected others.