lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ian Lea" <ian....@gmail.com>
Subject Re: CorruptIndexException with some versions of java
Date Tue, 18 Mar 2008 14:38:05 GMT
It's failed on servers running SuSE 10.0 and 8.2 (ancient!)

$ uname -a shows
Linux phoebe 2.6.13-15-smp #1 SMP Tue Sep 13 14:56:15 UTC 2005 x86_64
x86_64 x86_64 GNU/Linux

and

Linux phobos 2.4.20-64GB-SMP #1 SMP Mon Mar 17 17:56:03 UTC 2003 i686
unknown unknown GNU/Linux

The first one has a 2.8Ghz Intel CPU, don't know about the second.


I'll try and run the stress test.


--
Ian.



On Tue, Mar 18, 2008 at 2:17 PM, Yonik Seeley <yonik@apache.org> wrote:
>
> On Tue, Mar 18, 2008 at 7:38 AM, Ian Lea <ian.lea@gmail.com> wrote:
>  > Hi
>  >
>  >
>  >  When bulk loading into a new index I'm seeing this exception
>  >
>  >  Exception in thread "Thread-1"
>  >  org.apache.lucene.index.MergePolicy$MergeException:
>  >  org.apache.lucene.index.CorruptIndexException: doc counts differ for
>  >  segment _4l: fieldsReader shows 67861 but segmentInfo shows 67862
>  >         at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271)
>  >  Caused by: org.apache.lucene.index.CorruptIndexException: doc counts
>  >  differ for segment _4l: fieldsReader shows 67861 but segmentInfo shows
>  >  67862
>  >         at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:313)
>  >         at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
>  >         at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:221)
>  >         at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3093)
>  >         at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2834)
>  >         at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
>  >
>  >  when use java version 1.6.0_05-b13 or 1.6.0_04-b12 on linux, with
>  >  lucene 2.3.0 or 2.3.1 or lucene-core-2.3-SNAPSHOT from yesterday.
>  >
>  >  With java version 1.6.0_03-b05 things work fine.
>  >
>  >  The exception happens a few hundred thousand documents into the load.
>  >
>  >  A different program updating a different index with different data on
>  >  a different server gave a similar error on version 1.6.0_05-b13 and
>  >  lucene 2.3.0.
>  >
>  >  Any ideas?  Is this maybe a known issue or am I missing something obvious?
>
>  My guess is perhaps a thread safety bug, more likely in Lucene
>  indexing code (less likely in the JVM or specific libc).
>
>  What Linux version are you using?
>  What hardware are you running on (specifically, the CPU)?
>
>  If possible, it would be great if you could check out Lucene trunk,
>  crank up the iterations by modifying the TestStressIndexing2 and maybe
>  fiddle with some of the other parameters in
>  TestStressIndexing2.testMultiConfig(), and see if you can get it to
>  fail.
>
>
>  -Yonik
>
>  ---------------------------------------------------------------------
>
>
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>  For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message