lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rishisinghal <singhal.ri...@gmail.com>
Subject Indexer crashes with "hit exception during merge"
Date Thu, 13 Aug 2009 09:19:08 GMT

Hi,

I am trying to index documents and when all is complete and optimize is
called I get

IFD [main]: setInfoStream
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@4fced0
IW 0 [main]: setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/SYS$SYSDEVICE/RISHI/melon_1600
autoCommit=false mergePolicy=org
.apache.lucene.index.LogByteSizeMergePolicy@92cb49
mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@6038e4
ramBufferS
izeMB=16.0 maxBufferedDocs=5 maxBuffereDeleteTerms=-1 maxFieldLength=10000
index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0
 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]: optimize: index now _37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]:   flush: segment=null docStoreSegment=null docStoreOffset=0
flushDocs=false flushDeletes=true flushDocStores=false numD
ocs=0 numBufDelTerms=0
IW 0 [main]:   index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]: add merge to pendingMerges: _61:c245->_0 _9e:c485->_0
_ay:c225->_0 _az:c5->_0 _b0:c5->_0 [optimize] [total 1 pending]
IW 0 [main]: CMS: now merge
IW 0 [main]: CMS:   index: _37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]: CMS:   consider merge _61:c245->_0 _9e:c485->_0 _ay:c225->_0
_az:c5->_0 _b0:c5->_0 into _b5 [optimize]
IW 0 [main]: CMS:     launch new thread [Lucene Merge Thread #73]
IW 0 [main]: CMS:   no more merges pending; now return
IW 0 [Lucene Merge Thread #73]: CMS:   merge thread: start
IW 0 [Lucene Merge Thread #73]: now merge
  merge=_61:c245->_0 _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into
_b5 [optimize]
  merge=org.apache.lucene.index.MergePolicy$OneMerge@3725db
  index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 _ay:c225->_0
_az:c5->_0 _b0:c5->_0
IW 0 [Lucene Merge Thread #73]: merging _61:c245->_0 _9e:c485->_0
_ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize]
IW 0 [Lucene Merge Thread #73]: hit exception during merge
Exception in thread "Lucene Merge Thread #73"
org.apache.lucene.index.MergePolicy$MergeException:
java.io.FileNotFoundException: /SY
S$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
        at
org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309)
        at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286)
Caused by: java.io.FileNotFoundException:
/SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
        at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
        at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
        at
org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
        at
org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
        at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
        at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
        at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
        at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
Aug 13, 2009 2:40:56 PM org.hp.openvms.lucene.index.FileIndexerJSP jspMain
SEVERE: Indexing not completebackground merge hit exception: _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [
optimize]
java.io.IOException: background merge hit exception: _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize
]
        at
org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273)
        at
org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218)
        at
org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198)
        at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown
Source)
        at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source)
Caused by: java.io.FileNotFoundException:
/SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5))
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
        at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552)
        at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582)
        at
org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488)
        at
org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70)
        at
org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306)
        at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260)
        at
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220)
        at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884)
        at
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205)
        at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260)
IW 0 [main]: now flush at close
IW 0 [main]:   flush: segment=null docStoreSegment=null docStoreOffset=0
flushDocs=false flushDeletes=true flushDocStores=false numD
ocs=0 numBufDelTerms=0
IW 0 [main]:   index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]: CMS: now merge
IW 0 [main]: CMS:   index: _37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0
IW 0 [main]: CMS:   no more merges pending; now return
IW 0 [main]: now call final commit()
IW 0 [main]: startCommit(): start sizeInBytes=0
IW 0 [main]: startCommit index=_37:c465->_0 _4c:c165->_0 _61:c245->_0
_9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 changeCount=11
95
IW 0 [main]: now sync _37.cfs
IW 0 [main]: hit exception while closing
Exception in thread "main" java.io.FileNotFoundException:
/SYS$SYSDEVICE/RISHI/melon_1600/_37.cfs (i/o error (errno:5))
        at java.io.RandomAccessFile.open(Native Method)
        at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231)
        at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:456)
        at
org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4590)
        at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3369)
        at
org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1654)
        at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602)
        at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578)
        at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown
Source)
        at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source)



I have setMaxBufferedDocs = 5 and setMergeFactor=5. I was trying different
combinations of these to see if that helps but was unsuccessful. 


Regards,
Rishi
-- 
View this message in context: http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24951667.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message