lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Exceptions in merge thread (while optimizing) causing problems with subsequent reopens
Date Fri, 10 Apr 2009 00:05:53 GMT
These are serious corruption exceptions.

Is it at all possible two writers are accessing the index at the same time?

Can you describe more about how you're using Lucene?

Mike

On Thu, Apr 9, 2009 at 7:59 PM, Khawaja Shams <ksshams@gmail.com> wrote:
> Hello,
>  I am having a problem with reopening the IndexReader with Lucene 2.4 ( I
> updated to 2.4.1, but still no luck). The exception is preceded by an
> exception in optimizing the index. I am not reopening the reader while the
> commit or optimization is going on in the writer (optimizing happens in the
> same thread, but much less often). The issues go away once I turn off
> optimizations. I was also getting this problem before I turned off the use
> of compound files. I would appreciate any guidance.
>
> Thanks!
>
> Regards,
> Khawaja
>
>
> 2009-04-09 15:57:47,033 (941820) [Index Maint Thread] ERROR
> gov.nasa.ensemble.core.indexer.Indexer  - java.io.IOException: background
> merge hit exception: _8:C41258 _9:C11382 into _a [optimize]
> java.io.IOException: background merge hit exception: _8:C41258 _9:C11382
> into _a [optimize]
>    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273)
>    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218)
>    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198)
>    at
> gov.nasa.ensemble.core.indexer.Indexer$IndexMaintThread.run(Indexer.java:102)
> Caused by: org.apache.lucene.index.CorruptIndexException: doc counts differ
> for segment _8: fieldsReader shows 30074 but segmentInfo shows 41258
>    at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:362)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
>    at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
>    at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
> Exception in thread "Lucene Merge Thread #0"
> org.apache.lucene.index.MergePolicy$MergeException:
> org.apache.lucene.index.CorruptIndexException: doc counts differ for segment
> _8: fieldsReader shows 30074 but segmentInfo shows 41258
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
> Caused by: org.apache.lucene.index.CorruptIndexException: doc counts differ
> for segment _8: fieldsReader shows 30074 but segmentInfo shows 41258
>    at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:362)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
>    at
> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
>    at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
>    at
> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
> 2009-04-09 15:57:52,814 (947601) [Index Maint Thread] ERROR
> gov.nasa.ensemble.core.indexer.Indexer  - java.io.FileNotFoundException:
> /opt/users/merops/index/_a.fnm (No such file or directory)
> java.io.FileNotFoundException: /opt/users/merops/index/_a.fnm (No such file
> or directory)
>    at java.io.RandomAccessFile.open(Native Method)
>    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
>    at
> org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
>    at
> org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
>    at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
>    at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:482)
>    at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:58)
>    at
> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:341)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
>    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:269)
>    at
> org.apache.lucene.index.MultiSegmentReader.doReopen(MultiSegmentReader.java:201)
>    at
> org.apache.lucene.index.DirectoryIndexReader$2.doBody(DirectoryIndexReader.java:157)
>    at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:653)
>    at
> org.apache.lucene.index.DirectoryIndexReader.reopen(DirectoryIndexReader.java:179)
>    at
> gov.nasa.ensemble.core.indexer.Indexer$IndexMaintThread.run(Indexer.java:106)
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message