Re: lynx-dev -traversal -crawl gets into infinite loop

[Klaus Weide]

On Thu, 4 May 2000 address@hidden wrote:

> Hello Klaus,
> 
> Thanks for clarifying the delay involved. As it concerns a small
> attachment, I will forward this message I sent earlier to the
> lynx-dev address.

(still missing.)
(The very first link in his AbstractDeclarator.html file was
HREF="../../Yeti/YapiParser/AbstractDeclarator.html#AbstractDeclarator()".
I suspect a minimal HTML file with just that one link would show the
same problem.
His traverse.dat file contained just
http://localhost/Yeti/YapiParser/AbstractDeclarator.html
repeated over and over.
His traverse2.dat file contained just
http://localhost/Yeti/YapiParser/AbstractDeclarator.html         : Class  
AbstractDeclarator
repeated over and over.)


> Thanks,
> Paul
> 
> ---------------------- Forwarded by Paul van Loon_2
> 
> 
> Paul van Loon_2
> 26-04-2000 17:22
> To:   address@hidden@SMTP 
> cc:    
> Subject:      Re: lynx-dev -traversal -crawl gets into infinite loop  
> Classification:       Restricted
> 
> First of all, thanks for your fast response Klaus,
> 
> I'll try to give you detailed information about the problem.
> I'm using lynx on NT, -version reports:
> --
> Lynx Version 2.8.2rel.1 (01 Jun 1999)
> 
> Copyrights held by the University of Kansas, CERN, and other contributors.
> Distributed under the GNU General Public License.
> See http://lynx.browser.org/ and the online help for more information.
> --
> I'm starting it with:
> --
> c:\programs\lynx\lynx -trace -traversal -crawl 
> http://localhost/Yeti/YapiParser/AbstractDeclarator.html
> --
> this is a batch file that contains:
> --
> @ECHO OFF
> set term=vt100
> set temp=c:\temp
> set lynx_home=c:\programs\lynx
> set lynx_cfg=%lynx_home%\lynx.cfg
> %lynx_home%\lynxw32.exe %1 %2 %3 %4 %5
> --
> where lynxw32.exe is the renamed lynx.exe

(I'm not familiar with the details of setup for Windows; but this looks
all normal to me.)

> I got this win32 version from:
> --
> http://www.fdisk.com/doslynx/lynxport.htm
> --

> When I executed lynx, it will keep fetching the same page over and
> over again until i kill it.  It then has generated a bunch of .dat
> files. These are included in this post, together with the
> AbstractDeclarator.html file. This html file is generated by javadoc
>
> I think this should allow you to reproduce the problem.  (I used a
> locally running Apache server to access the .html file, if you could
> allow -traversal -crawl to accept the "file:" protocol too, I would
> be grateful)

[I'm not personally interested in extending the -traversal functionality;
but I'm sure that, if someone sends a patch do do that, with all the
necessary changes including documentation, he will consider it...

Well, it's possible to *fake* "http:" when the actual "physical" access
is through the "file:" protocol.  For your example, something like:
RULE:Map http://localhost/Yeti/* file://localhost/var/www/Yeti/*
would make the document(s) *appear* to be under http://localhost/Yeti/
(as far as most of lynx is concerned, including what appears in the
traverse* file), while they are actuallly accessed from disk (without
involving any HTTP requests, thus no web server needed).]

I reproduced your setup on linux: the file AbstractDeclarator.html you
sent, served by a local apache as exactly the same URL
(http://localhost/Yeti/YapiParser/AbstractDeclarator.html).

I did not reproduce your problem.  Both lynx 2.8.2 and lynx (nearly-)2.8.3
did not loop endlessly, but traversed the document and terminated as
expected.

So, most likely this is a problem specific to Win32 Lynx (and maybe DOS 386
Lynx), or only some binaries.

- Try other binaries.  There are pointers at
<http://www.fdisk.com/doslynx/lynxport.htm> and <http://lynx.isc.org/release/>
and <http://lynx.isc.org/current/#w32>.
The problem may have already been fixed.
- Ask on the list whether -traversal does work at all for anyone under Win32
(and in which version).

I have an idea that this problem could be (could have been?) caused by
differences in line-end convention (\n vs. \r\n), in code like the
following (LYTraversal.c):

    while (LYSafeGets(&buffer, ifp) != NULL) {
        if (STREQ(line,buffer)) {
            result = TRUE;
            break;
        }
    } /* end while */

but have no way of testing this for Windows.

   Klaus



;
; To UNSUBSCRIBE:  Send a mail message to address@hidden
;                  with "unsubscribe lynx-dev" (without the
;                  quotation marks) on a line by itself.
;