Mailing-List: contact nutch-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: nutch-user@incubator.apache.org
Received-SPF: neutral (hermes.apache.org: 209.226.175.97 is neither permitted
 nor denied by domain of emilijan@cpuedge.com)
Message-ID: <4284EFE6.2050109@cpuedge.com>
Date: Fri, 13 May 2005 14:20:22 -0400
From: EM <emilijan@cpuedge.com>
User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206)
MIME-Version: 1.0
To: nutch-user@incubator.apache.org
Subject: Re: How does this sound
References: <a276dbda05051310547522b389@mail.gmail.com>
In-Reply-To: <a276dbda05051310547522b389@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Sounds fine with me although more experience people here may have 
different opinion.

One small thing, if you are setting up each site individually, then, 
fully disable the spidering. That way, you can inject individual sites 
by yourself.

Good luck,
Emilijan
Ian Reardon wrote:

>I am going to crawl a small set of sites and I never want to go off
>site and I also want to strictly control my link dept.
>
>I setup crawls for each site using the crawl command.  Then manually
>move the segments folder to my "master" directory and re-index.  (This
>can all be scripted).  This gives me the flex ability to QA each
>individual crawl.
>
>Am I jumping through unnecessary hoops here or does this sound like a
>reasonable plan?
>  
>