lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jérôme Thièvre <jthie...@ina.fr>
Subject Re: merging Parallel indexes (can indexWriter.addIndexesNoOptimize be used?)
Date Wed, 04 Nov 2009 13:24:57 GMT
Hello Geert-Jan,

it's possible to merge several parallel physical indexes (viewed as one 
logical index with a ParallelReader).
Just use the method IndexWriter.addIndexes(IndexReader[] readers):

IndexReader[]  physicalReaders = ...; // Your readers here

IndexWriter iw = new IndexWriter(...);
ParallelReader parallelReader = new ParallelReader(true);

for(IndexReader reader : physicalReaders)
parallelReader .add(reader);


IndexReader[] readersToMerge = { parallelReader };
iw.addIndexes(readersToMerge);
// Optional optimize
iw.optimize();
iw.close();
parallelReader.close();


I used this trick to deploy an index made of 4 parallel indexes. Using 
ParallelReader on large indexes is not very efficient for searching , 
but I need to use parallel indexes because one of them is huge but 
evolve very slowly compared to the others.

Jérôme

Britske a écrit :
> Given two parallel indexes which contain the same products but different
> fields, one with slowly changing fields and one with fields which are
> updated regularly: 
>
> Is it possible to periodically merge these to form a single index?  (thereby
> representing a frozen snapshot in time) 
>
> For example: Can indexWriter.addIndexesNoOptimize handle this, or was it
> (only) designed for merging shards? 
> If not, is there another option (3rd party or not) to use, or would I have
> to resort to low-level hacking? 
>
> Thanks, 
> Geert-Jan
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message