hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki ...@getopt.org>
Subject Re: Hadoop + Lucene integration: possible? how?
Date Mon, 15 Jan 2007 12:53:59 GMT
maarten@sherpa-consulting.be wrote:
> I'm new to lucene and Hadoop but what I can't seem to find in the 
> docs, internet... is how (and if possible?) to use Hadoop as the 
> underlying FS for Lucene?
> Could anyone explain me how these can be tied together? Some small 
> code/configuration example would be nice :-)

It's possible to use Hadoop DFS to host a read-only Lucene index and use 
it for searching (Nutch has an implementation of FSDirectory for this 
purpose), but the performance is not stellar ... Currently it's not 
(yet) possible to use HDFS for creating Lucene indexes, a minor change 
to Lucene index format would be required.

Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

View raw message