hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <tdunn...@maprtech.com>
Subject Re: Hadoop/Lucene + Solr architecture suggestions?
Date Wed, 10 Oct 2012 14:58:57 GMT
I prefer to create indexes in the reducer personally. 

Also you can avoid the copies if you use an advanced hadoop-derived distro. Email me off list
for details. 

Sent from my iPhone

On Oct 9, 2012, at 7:47 PM, Mark Kerzner <mark.kerzner@shmsoft.com> wrote:

> Hi,
> if I create a Lucene index in each mapper, locally, then copy them to under /jobid/mapid1,
/jodid/mapid2, and then in the reducers copy them to some Solr machine (perhaps even merging),
does such architecture makes sense, to create a searchable index with Hadoop?
> Are there links for similar architectures and questions?
> Thank you. Sincerely,
> Mark

View raw message