lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Performance of never optimizing
Date Wed, 05 Nov 2008 16:33:31 GMT

Otis Gospodnetic wrote:

>> Our current default behaviour is a merge factor of 4. We perform an  
>> optimization
>> on the index every 4000 additions. We also perform an optimize at  
>> midnight. Our
>
> I wouldn't optimize every 4000 additions - you are killing IO,  
> rewriting the whole index, while trying to provide fast searches,  
> plus you are locking the index for other modifications.


One small clarification: optimize can run in the BG.  It doesn't block
other IndexWriter operations.  EG you can continue adding & deleting
docs.  Optimize() just guarantees that those segments that existed at
the start will be merged together.  Other segments that are flushed
after optimize had started will not be merged.

Of course, optimize is tremendously IO intensive so this may still
block out searches if the performance becomes hideously bad because
the IO system is saturated.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message