lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christiaan Fluit <>
Subject Re: semi-infinite loop during merging
Date Fri, 24 Apr 2009 21:02:00 GMT
Michael McCandless wrote:
>> - even though the commitMerge returns false, it should probably not get into
>> an infinite loop. Is this an internal Lucene problem or is there something I
>> can/should do about it myself?
> Yes, something is wrong with Lucene's handling of OOME.  It certainly
> should not lead to infinite merge attempts.  I'll dig (once back from
> vacation) to see if I can find this path.  Likely we need to prevent
> launching of new merges after an OOME.  I think you must've happened
> to hit OOME when a merge was running.

I have some more info.

I added message(String) invocations in all places where the IW.hitOOM 
flag is set, to see which method turns it on. It turned out to be 
addDocument (twice). These OOME's only happen with the JET build, which 
explains why the Java build does not show the exploding index behavior: 
the hitOOM flag is simply never set and the merge is allowed to proceed 

The flag is definitely not set while the IW is merging, nor do any 
OOME's appear in my log files during merging. Therefore, there must be a 
problem in how the merge operation responds to the flag being set.

Rollback does not work for me, as my IW is in auto-commit mode. It gives 
an IllegalStateException when I invoke it.

A workaround that does work for me is to close and reopen the 
IndexWriter immediately after an OOME occurs.

Let me know if I can be of any more help.



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message