Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of ksshams@gmail.com designates
 209.85.146.179 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=WhE31wIqmtU8NsPLj7EPBZGd1ClAy0BVC36qnzGHgTMhn2ch6s3BsBMIPWy/NxlZLY
         dNoG8pbQHfsSriE3SzNXL7lJX7PbsidUFyisgER8Tyj/lG3hKGwSZXxUG+srJHZtV3De
         KAvAXlXXDNsgaLuXhoWrgh4x9TT1QcwqdtcGo=
MIME-Version: 1.0
Date: Thu, 9 Apr 2009 16:59:26 -0700
Message-ID: <6d6cf45b0904091659se193b03l9c5344e59fa05dc4@mail.gmail.com>
Subject: Exceptions in merge thread (while optimizing) causing problems with
	subsequent reopens
From: Khawaja Shams <ksshams@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=0016364585fcc1743f046728097c

--0016364585fcc1743f046728097c
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Hello,
  I am having a problem with reopening the IndexReader with Lucene 2.4 ( I
updated to 2.4.1, but still no luck). The exception is preceded by an
exception in optimizing the index. I am not reopening the reader while the
commit or optimization is going on in the writer (optimizing happens in the
same thread, but much less often). The issues go away once I turn off
optimizations. I was also getting this problem before I turned off the use
of compound files. I would appreciate any guidance.

Thanks!

Regards,
Khawaja


2009-04-09 15:57:47,033 (941820) [Index Maint Thread] ERROR
gov.nasa.ensemble.core.indexer.Indexer  - java.io.IOException: background
merge hit exception: _8:C41258 _9:C11382 into _a [optimize]
java.io.IOException: background merge hit exception: _8:C41258 _9:C11382
into _a [optimize]
    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273)
    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218)
    at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198)
    at
gov.nasa.ensemble.core.indexer.Indexer$IndexMaintThread.run(Indexer.java:102)
Caused by: org.apache.lucene.index.CorruptIndexException: doc counts differ
for segment _8: fieldsReader shows 30074 but segmentInfo shows 41258
    at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:362)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
    at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
    at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
    at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
    at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
Exception in thread "Lucene Merge Thread #0"
org.apache.lucene.index.MergePolicy$MergeException:
org.apache.lucene.index.CorruptIndexException: doc counts differ for segment
_8: fieldsReader shows 30074 but segmentInfo shows 41258
    at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
    at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
Caused by: org.apache.lucene.index.CorruptIndexException: doc counts differ
for segment _8: fieldsReader shows 30074 but segmentInfo shows 41258
    at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:362)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
    at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
    at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
    at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
    at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
2009-04-09 15:57:52,814 (947601) [Index Maint Thread] ERROR
gov.nasa.ensemble.core.indexer.Indexer  - java.io.FileNotFoundException:
/opt/users/merops/index/_a.fnm (No such file or directory)
java.io.FileNotFoundException: /opt/users/merops/index/_a.fnm (No such file
or directory)
    at java.io.RandomAccessFile.open(Native Method)
    at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
    at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
    at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
    at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
    at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:482)
    at org.apache.lucene.index.FieldInfos.<init>(FieldInfos.java:58)
    at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:341)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:269)
    at
org.apache.lucene.index.MultiSegmentReader.doReopen(MultiSegmentReader.java:201)
    at
org.apache.lucene.index.DirectoryIndexReader$2.doBody(DirectoryIndexReader.java:157)
    at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:653)
    at
org.apache.lucene.index.DirectoryIndexReader.reopen(DirectoryIndexReader.java:179)
    at
gov.nasa.ensemble.core.indexer.Indexer$IndexMaintThread.run(Indexer.java:106)

--0016364585fcc1743f046728097c--