lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Britske <gbr...@gmail.com>
Subject Re: merging Parallel indexes (can indexWriter.addIndexesNoOptimize be used?)
Date Wed, 04 Nov 2009 13:41:53 GMT

Yeah excellent! This should indeed work!

Thanks, 
Geert-Jan


Jérôme Thièvre wrote:
> 
> Hello Geert-Jan,
> 
> it's possible to merge several parallel physical indexes (viewed as one 
> logical index with a ParallelReader).
> Just use the method IndexWriter.addIndexes(IndexReader[] readers):
> 
> IndexReader[]  physicalReaders = ...; // Your readers here
> 
> IndexWriter iw = new IndexWriter(...);
> ParallelReader parallelReader = new ParallelReader(true);
> 
> for(IndexReader reader : physicalReaders)
> parallelReader .add(reader);
> 
> 
> IndexReader[] readersToMerge = { parallelReader };
> iw.addIndexes(readersToMerge);
> // Optional optimize
> iw.optimize();
> iw.close();
> parallelReader.close();
> 
> 
> I used this trick to deploy an index made of 4 parallel indexes. Using 
> ParallelReader on large indexes is not very efficient for searching , 
> but I need to use parallel indexes because one of them is huge but 
> evolve very slowly compared to the others.
> 
> Jérôme
> 
> Britske a écrit :
>> Given two parallel indexes which contain the same products but different
>> fields, one with slowly changing fields and one with fields which are
>> updated regularly: 
>>
>> Is it possible to periodically merge these to form a single index? 
>> (thereby
>> representing a frozen snapshot in time) 
>>
>> For example: Can indexWriter.addIndexesNoOptimize handle this, or was it
>> (only) designed for merging shards? 
>> If not, is there another option (3rd party or not) to use, or would I
>> have
>> to resort to low-level hacking? 
>>
>> Thanks, 
>> Geert-Jan
>>   
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/merging-Parallel-indexes-%28can-indexWriter.addIndexesNoOptimize-be-used-%29-tp26161322p26196481.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message