nutch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Zhou <jefferson.z...@gmail.com>
Subject Re: Is it possible to break the fetch process into multiple processes?
Date Mon, 06 Dec 2010 04:42:16 GMT
Thanks Paul. You are very helpful!
Jeff

On Sun, Dec 5, 2010 at 7:31 PM, Paul Dhaliwal <subpaul@gmail.com> wrote:

> Nutch tutorial will show you how to inject urls and crawl using the
> injected
> urls.
>
> I did not use hadoop but I did run my crawls on lot of machines and each
> machine had multiple  crawl processes running.
>
> Paul
>
> On Dec 5, 2010 3:59 PM, "Jeff Zhou" <jefferson.zhou@gmail.com> wrote:
> Hi Paul, how do you assign url folders to each or more of the fetch
> threads?
> Does this involve Hadoop?
> Jeff
>
>
> On Sun, Dec 5, 2010 at 5:53 PM, Paul Dhaliwal <subpaul@gmail.com> wrote:
>
> > Hi jeff,
> >
> > Yes, I do ...
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message