lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Way to repair an index broking during 1/2 optimize?
Date Fri, 09 Jul 2004 16:03:52 GMT
Kevin A. Burton wrote:
>> With the typical handful of fields, one should never see more than 
>> hundreds of files.
>>
> We only have 13 fields... Though to be honest I'm worried that even if I 
> COULD do the optimize that it would run out of file handles.

Optimization doesn't open all files at once.  The most files that are 
ever opened by an IndexWriter is just:

4 + (5 + numIndexedFields) * (mergeFactor-1)

This includes during optimization.

However, when searching, an IndexReader must keep most files open.  In 
particular, the maximum number of files an unoptimized, non-compound 
IndexReader can have open is:

(5 + numIndexedFields) * (mergeFactor-1) * 
(log_base_mergeFactor(numDocs/minMergeDocs))

A compound IndexReader, on the other hand, should open at most, just:

(mergeFactor-1) * (log_base_mergeFactor(numDocs/minMergeDocs))

An optimized, non-compound IndexReader will open just (5 + 
numIndexedFields) files.

And an optimized, compound IndexReader should only keep one file open.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org


Mime
View raw message