hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: map/reduce and Lucene integration question
Date Thu, 13 Dec 2007 19:31:23 GMT
Ted Dunning wrote:
> I don't think so (but I don't run nutch)
> 
> To actually run searches, the search engines copy the index to local
> storage.  Having them in HDFS is very nice, however, as a way to move them
> to the right place.

Nutch can search in Lucene indexes on HDFS (see 
org.apache.nutch.indexer.FsDirectory). However, the performance hit is 
substantial, so the best practice is to copy indexes to a local FS.


-- 
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com


Mime
View raw message