lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: CorruptIndexException with some versions of java
Date Mon, 24 Mar 2008 09:08:52 GMT

Spookily, I believe this is in fact some sort of JVM hotspot compiler  
bug.  Adding -Xbatch (forces up-front compilation) prevents (works  
around) the problem.

I added a counter, to IndexOutput, of total bytes written, and then  
in SegmentMerger.mergeFields, I added an assert that the #docs  
written through FieldsWriter matched the number of bytes written to  
the fdx file.  Insanely, that assert would fail when the issue was  
hit, which is just not possible from the code.  It's as if when the  
BG compilation finished and the native method got swapped into the  
JVM, it somehow "missed" a call to IndexOutput.writeLong.

I committed some of these asserts into SegmentMerger on both 2.3 &  

I'll respond on the java-user thread to bring closure here.


Yonik Seeley wrote:
> On Wed, Mar 19, 2008 at 1:18 PM, Michael McCandless
> <> wrote:
>>  I'm CC'ing java-dev. To sum up: sometimes when we merge fields, the
>>  fdx file ends up exactly one document too short.  In adding numerous
>>  asserts around this code in, insanely, somehow  
>> the
>>  call to indexStream.writeLong fails to actually "happen" when we  
>> call
>>  FieldsWriter.addDocument, even though from looking at the code I see
>>  no way to explain that.  As hard as it is to believe, it really is
>>  looking like a strange JVM bug at this point...
> JDK 1.6_04 was a major upgrade of hotspot.
> Is there a way to dump the compiled machine code that hotspot
> eventually generates for that method?
> -Yonik
>>  Has anyone else seen any odd behavior on update 4 or 5 of JDK 1.6?
>>  The issue does not happen on previous updates of JDK 1.6.
>>  Mike
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message