lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lucene Merge failing on Open Files
Date Wed, 06 Apr 2011 10:44:51 GMT
Can you turn on IndexWriter's infoStream, get the failure to happen,
and post the resulting output?

How are you adding the multiple indices together?  Can you post the
code that does that?

The number of open file handles needed during indexing is a function
of how many merges are running and how large (how many segments) each
of those merges are.  Lucene in Action 2, chapter 11, has a cool graph
showing this... (and also file handle usage by IndexReader, reopening
periodically).

Mike

http://blog.mikemccandless.com

On Wed, Apr 6, 2011 at 4:50 AM, Paul Taylor <paul_t100@fastmail.fm> wrote:
> On 04/04/2011 21:06, Simon Willnauer wrote:
>>
>> On Mon, Apr 4, 2011 at 9:59 PM, Paul Taylor<paul_t100@fastmail.fm>  wrote:
>>>
>>> On 04/04/2011 20:13, Michael McCandless wrote:
>>>>
>>>> How are you merging these indices?  (IW.addIndexes?).
>>>>
>>>> Are you changing any of IW's defaults, eg mergeFactor?
>>>>
>>>> Mike
>>>
>>> Hi Mike
>>>
>>> I have
>>>
>>> indexWriter.setMaxBufferedDocs(10000);
>>> indexWriter.setMergeFactor(3000);
>>>
>> I didn't read though the entire email but MergeFactor 3000 doesn't
>> look right at all. You should try something between 5 and 30. I
>> haven't seen a mergeFactor>  50 doing any good in a common env. Why do
>> you use such a large factor?
>>
>> simon
>>
> decreased merge factor to 10, and it still fails
> Is there way I can find out how many open file handles lucene will need ?
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message