nutch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Reardon <>
Subject How does this sound
Date Fri, 13 May 2005 17:54:31 GMT
I am going to crawl a small set of sites and I never want to go off
site and I also want to strictly control my link dept.

I setup crawls for each site using the crawl command.  Then manually
move the segments folder to my "master" directory and re-index.  (This
can all be scripted).  This gives me the flex ability to QA each
individual crawl.

Am I jumping through unnecessary hoops here or does this sound like a
reasonable plan?

View raw message