nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: RSS-fecter and index individul-how can i realize this function
Date Wed, 07 Feb 2007 17:58:52 GMT
Renaud Richardet wrote:
> I see. I was thinking that I could index the feed items without having 
> to fetch them individually.

Okay, so if Parser#parse returned a Map<String,Parse>, then the URL for 
each parse should be that of its link, since you don't want to fetch 
that separately.  Right?

So now the question is, how much impact would this change to the Parser 
API have on the rest of Nutch?  It would require changes to all Parser 
implementations, to ParseSegement, to ParseUtil, and to Fetcher.  But, 
as far as I can tell, most of these changes look straightforward.

Doug

Mime
View raw message