perl-docs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mer...@stonehenge.com (Randal L. Schwartz)
Subject Re: status update
Date Sun, 03 Mar 2002 14:25:53 GMT
>>>>> "Bill" == Bill Moseley <moseley@hank.org> writes:

Bill> It takes a while (on my machine instead of indexing the site in
Bill> less then a minute it takes about 18 minutes to check all the
Bill> links.

Bill> Some don't make much sense, check out the www.modperl.com
Bill> error?? -- I'll look into it next week.  Again, it's just doing
Bill> simple HEAD requests with LWP.  Clearly a lot of those work in
Bill> browsers, but they are returning errors here.

If you need a decent spidering linkchecker with more tweaking configuration
settings and parallel processing power than is reasonable (:-), check
<http://www.stonehenge.com/merlyn/LinuxMag/col16.html>, downloadable
from <http://www.stonehenge.com/merlyn/LinuxMag/col16.listing.txt>.
It retries bad "HEAD" requests as "GET", for example.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Perl/Unix/security consulting, Technical writing, Comedy, etc. etc.
See PerlTraining.Stonehenge.com for onsite and open-enrollment Perl training!

---------------------------------------------------------------------
To unsubscribe, e-mail: docs-dev-unsubscribe@perl.apache.org
For additional commands, e-mail: docs-dev-help@perl.apache.org


Mime
View raw message