lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wojtek212 <wojtekg...@gmail.com>
Subject Re: FileNotFoundException during indexing
Date Fri, 01 Aug 2008 14:17:56 GMT

Here is Lucene log with first exceptions that occured (FSDirectory with
NativeFSLockFactory).

IFD [Thread-79]: setInfoStream
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@1a00d25
IW 4 [Thread-79]: setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/tmp/content/3615.0-3618.0
autoCommit=true
mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@bf033a
mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@6cf504
ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1
maxFieldLength=10000 index=_1:c28 _2:c1
IW 4 [Thread-79]: optimize: index now _1:c28 _2:c1
IW 4 [Thread-79]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=true flushDocStores=false
numDocs=0 numBufDelTerms=1
IW 4 [Thread-79]:   index before flush _1:c28 _2:c1
IW 4 [Thread-79]: flush 1 buffered deleted terms and 0 deleted docIDs on 2
segments.
IW 4 [Thread-79]: checkpoint: wrote segments file "segments_9"
IFD [Thread-79]: now checkpoint "segments_9" [2 segments ; isCommit = true]
IFD [Thread-79]: deleteCommits: now remove commit "segments_8"
IFD [Thread-79]: delete "segments_8"
IW 4 [Thread-79]: LMP: findMerges: 2 segments
IW 4 [Thread-79]: LMP:   level -1.0 to 4.1957335: 2 segments
IW 4 [Thread-79]: CMS: now merge
IW 4 [Thread-79]: CMS:   index: _1:c28 _2:c1
IW 4 [Thread-79]: CMS:   no more merges pending; now return
IW 4 [Thread-79]: add merge to pendingMerges: _1:c28 _2:c1 [optimize] [total
1 pending]
IW 4 [Thread-79]: CMS: now merge
IW 4 [Thread-79]: CMS:   index: _1:c28 _2:c1
IW 4 [Thread-79]: CMS:   consider merge _1:c28 _2:c1 into _3 [optimize]
IW 2 [Thread-80]: CMS:   merge thread: done
IW 4 [Thread-79]: CMS:     launch new thread [Thread-82]
IW 4 [Thread-79]: CMS:   no more merges pending; now return
IW 4 [Thread-82]: CMS:   merge thread: start
IW 4 [Thread-82]: now merge
  merge=_1:c28 _2:c1 into _3 [optimize]
  index=_1:c28 _2:c1
IW 4 [Thread-82]: merging _1:c28 _2:c1 into _3 [optimize]
IW 4 [Thread-82]: merge: total 28 docs
IW 4 [Thread-82]: commitMerge: _1:c28 _2:c1 into _3 [optimize]
IW 4 [Thread-82]: commitMerge _1:c28 _2:c1 into _3 [optimize]
IW 4 [Thread-82]: checkpoint: wrote segments file "segments_a"
IFD [Thread-82]: now checkpoint "segments_a" [1 segments ; isCommit = true]
IFD [Thread-82]: deleteCommits: now remove commit "segments_9"
IFD [Thread-82]: delete "segments_9"
IFD [Thread-82]: delete "_1.cfs"
IFD [Thread-82]: delete "_1_1.del"
IFD [Thread-82]: delete "_2.cfs"
IW 4 [Thread-82]: checkpoint: wrote segments file "segments_b"
IFD [Thread-82]: now checkpoint "segments_b" [1 segments ; isCommit = true]
IFD [Thread-82]: deleteCommits: now remove commit "segments_a"
IFD [Thread-82]: delete "segments_a"
IFD [Thread-82]: delete "_3.fnm"
IFD [Thread-82]: delete "_3.frq"
IFD [Thread-82]: delete "_3.prx"
IFD [Thread-82]: delete "_3.tis"
IFD [Thread-82]: delete "_3.tii"
IFD [Thread-82]: delete "_3.nrm"
IFD [Thread-82]: delete "_3.fdx"
IFD [Thread-82]: delete "_3.fdt"
IW 4 [Thread-82]: CMS:   merge thread: done
IW 4 [Thread-79]: now flush at close
IW 4 [Thread-79]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=false flushDocStores=false
numDocs=0 numBufDelTerms=0
IW 4 [Thread-79]:   index before flush _3:c28
IW 4 [Thread-79]: CMS: now merge
IW 4 [Thread-79]: CMS:   index: _3:c28
IW 4 [Thread-79]: CMS:   no more merges pending; now return
IW 4 [Thread-79]: at close: _3:c28
IFD [Thread-79]: setInfoStream
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@dd0e8a
IW 5 [Thread-79]: setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/tmp/content/3615.0-3618.0
autoCommit=true
mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@e5a644
mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@106cc42
ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1
maxFieldLength=10000 index=_3:c28
IW 5 [Thread-79]: now flush at close
IW 5 [Thread-79]:   flush: segment=_4 docStoreSegment=_4 docStoreOffset=0
flushDocs=true flushDeletes=false flushDocStores=true numDocs=1
numBufDelTerms=0
IW 5 [Thread-79]:   index before flush _3:c28

flush postings as segment _4 numDocs=1

closeDocStore: 2 files to flush to segment _4 numDocs=1
  oldRAMSize=81920 newFlushedSize=847 docs/MB=1,237.988 new/old=1.034%
IW 5 [Thread-79]: checkpoint: wrote segments file "segments_c"
IFD [Thread-79]: now checkpoint "segments_c" [2 segments ; isCommit = true]
IFD [Thread-79]: deleteCommits: now remove commit "segments_b"
IFD [Thread-79]: delete "segments_b"
IW 5 [Thread-79]: checkpoint: wrote segments file "segments_d"
IFD [Thread-79]: now checkpoint "segments_d" [2 segments ; isCommit = true]
IFD [Thread-79]: deleteCommits: now remove commit "segments_c"
IFD [Thread-79]: delete "segments_c"
IFD [Thread-79]: delete "_4.fnm"
IFD [Thread-79]: delete "_4.frq"
IFD [Thread-79]: delete "_4.prx"
IFD [Thread-79]: delete "_4.tis"
IFD [Thread-79]: delete "_4.tii"
IFD [Thread-79]: delete "_4.nrm"
IFD [Thread-79]: delete "_4.fdx"
IFD [Thread-79]: delete "_4.fdt"
IW 5 [Thread-79]: LMP: findMerges: 2 segments
IW 5 [Thread-79]: LMP:   level -1.0 to 4.197694: 2 segments
IW 5 [Thread-79]: CMS: now merge
IW 5 [Thread-79]: CMS:   index: _3:c28 _4:c1
IW 5 [Thread-79]: CMS:   no more merges pending; now return
IW 5 [Thread-79]: CMS: now merge
IW 5 [Thread-79]: CMS:   index: _3:c28 _4:c1
IW 5 [Thread-79]: CMS:   no more merges pending; now return
IW 5 [Thread-79]: at close: _3:c28 _4:c1
IFD [Thread-85]: setInfoStream
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@1498cd
IW 6 [Thread-85]: setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/tmp/content/3615.0-3618.0
autoCommit=true
mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@3be7e8
mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@f64915
ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1
maxFieldLength=10000 index=_3:c28 _4:c1
IW 6 [Thread-85]: optimize: index now _3:c28 _4:c1
IW 6 [Thread-85]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=true flushDocStores=false
numDocs=0 numBufDelTerms=1
IW 6 [Thread-85]:   index before flush _3:c28 _4:c1
IW 6 [Thread-85]: flush 1 buffered deleted terms and 0 deleted docIDs on 2
segments.
IW 6 [Thread-85]: checkpoint: wrote segments file "segments_e"
IFD [Thread-85]: now checkpoint "segments_e" [2 segments ; isCommit = true]
IFD [Thread-85]: deleteCommits: now remove commit "segments_d"
IFD [Thread-85]: delete "segments_d"
IW 6 [Thread-85]: LMP: findMerges: 2 segments
IW 6 [Thread-85]: LMP:   level -1.0 to 4.1980243: 2 segments
IW 6 [Thread-85]: CMS: now merge
IW 6 [Thread-85]: CMS:   index: _3:c28 _4:c1
IW 6 [Thread-85]: CMS:   no more merges pending; now return
IW 6 [Thread-85]: add merge to pendingMerges: _3:c28 _4:c1 [optimize] [total
1 pending]
IW 6 [Thread-85]: CMS: now merge
IW 6 [Thread-85]: CMS:   index: _3:c28 _4:c1
IW 6 [Thread-85]: CMS:   consider merge _3:c28 _4:c1 into _5 [optimize]
IW 6 [Thread-85]: CMS:     launch new thread [Thread-89]
IW 6 [Thread-85]: CMS:   no more merges pending; now return
IW 6 [Thread-89]: CMS:   merge thread: start
IW 6 [Thread-89]: now merge
  merge=_3:c28 _4:c1 into _5 [optimize]
  index=_3:c28 _4:c1
IW 6 [Thread-89]: merging _3:c28 _4:c1 into _5 [optimize]
IW 6 [Thread-89]: merge: total 28 docs
IW 42 [Thread-86]: optimize: index now _3:c28 _4:c1
IW 42 [Thread-86]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=true flushDocStores=false
numDocs=0 numBufDelTerms=1
IW 42 [Thread-86]:   index before flush _3:c28 _4:c1
IW 42 [Thread-86]: flush 1 buffered deleted terms and 0 deleted docIDs on 2
segments.
IW 42 [Thread-86]: checkpoint: wrote segments file "segments_f"
IFD [Thread-86]: now checkpoint "segments_f" [2 segments ; isCommit = true]
IFD [Thread-86]: deleteCommits: now remove commit "segments_e"
IFD [Thread-86]: delete "segments_e"
IFD [Thread-86]: delete "_3_1.del"
IW 42 [Thread-86]: LMP: findMerges: 2 segments
IW 42 [Thread-86]: LMP:   level -1.0 to 4.1980243: 2 segments
IW 42 [Thread-86]: CMS: now merge
IW 42 [Thread-86]: CMS:   index: _3:c28 _4:c1
IW 42 [Thread-86]: CMS:   no more merges pending; now return
IW 42 [Thread-86]: add merge to pendingMerges: _3:c28 _4:c1 [optimize]
[total 1 pending]
IW 42 [Thread-86]: CMS: now merge
IW 42 [Thread-86]: CMS:   index: _3:c28 _4:c1
IW 42 [Thread-86]: CMS:   consider merge _3:c28 _4:c1 into _5 [optimize]
IW 6 [Thread-89]: commitMerge: _3:c28 _4:c1 into _5 [optimize]
IW 6 [Thread-89]: commitMerge _3:c28 _4:c1 into _5 [optimize]
IW 6 [Thread-89]: hit exception creating merged deletes file
IFD [Thread-89]: refresh [prefix=_5]: removing newly created unreferenced
file "_5.nrm"
IFD [Thread-89]: delete "_5.nrm"
IW 6 [Thread-89]: hit exception during merge
Exception in thread "Thread-89"
org.apache.lucene.index.MergePolicy$MergeException:
java.io.FileNotFoundException: /tmp/content/3615.0-3618.0/_3_1.del (No such
file or directory)
	at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:271)
Caused by: java.io.FileNotFoundException:
/tmp/content/3615.0-3618.0/_3_1.del (No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:506)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:536)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:440)
	at org.apache.lucene.util.BitVector.<init>(BitVector.java:180)
	at org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:2827)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3295)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
	at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
IW 42 [Thread-86]: CMS:     launch new thread [Thread-93]
IW 6 [Thread-85]: now flush at close
IW 6 [Thread-85]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=false flushDocStores=false
numDocs=0 numBufDelTerms=0
IW 6 [Thread-85]:   index before flush _3:c28 _4:c1
IW 6 [Thread-85]: CMS: now merge
IW 6 [Thread-85]: CMS:   index: _3:c28 _4:c1
IW 6 [Thread-85]: CMS:   no more merges pending; now return
IW 6 [Thread-85]: close: wrote segments file "segments_f"
IFD [Thread-85]: now checkpoint "segments_f" [2 segments ; isCommit = true]
IFD [Thread-85]: deleteCommits: now remove commit "segments_e"
IFD [Thread-85]: delete "segments_e"
IW 6 [Thread-85]: at close: _3:c28 _4:c1
IFD [Thread-81]: setInfoStream
deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@1ddcef0
IW 7 [Thread-81]: setInfoStream:
dir=org.apache.lucene.store.FSDirectory@/tmp/content/3615.0-3618.0
autoCommit=true
mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@13d2d11
mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@159e0c2
ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1
maxFieldLength=10000 index=_3:c28 _4:c1
IW 7 [Thread-81]: optimize: index now _3:c28 _4:c1
IW 7 [Thread-81]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=true flushDocStores=false
numDocs=0 numBufDelTerms=1
IW 7 [Thread-81]:   index before flush _3:c28 _4:c1
IW 7 [Thread-81]: flush 1 buffered deleted terms and 0 deleted docIDs on 2
segments.
IW 7 [Thread-81]: hit exception flushing segment null
IFD [Thread-81]: now checkpoint "segments_f" [2 segments ; isCommit = false]
IW 7 [Thread-81]: now flush at close
IW 7 [Thread-81]:   flush: segment=null docStoreSegment=null
docStoreOffset=0 flushDocs=false flushDeletes=true flushDocStores=false
numDocs=0 numBufDelTerms=1
IW 7 [Thread-81]:   index before flush _3:c28 _4:c1
IW 7 [Thread-81]: flush 1 buffered deleted terms and 0 deleted docIDs on 2
segments.
IW 7 [Thread-81]: hit exception flushing segment null
IFD [Thread-81]: now checkpoint "segments_f" [2 segments ; isCommit = false]
IW 7 [Thread-81]: hit exception while closing
Exception in thread "Thread-85" java.lang.RuntimeException:
com.test.search.IndexingException: java.io.IOException: background merge hit
exception: _3:c28 _4:c1 into _5 [optimize]
	at
com.test.vcssearch.DefaultServiceIndexer$2.run(DefaultServiceIndexer.java:245)
	at java.lang.Thread.run(Thread.java:595)
Caused by: com.test.search.IndexingException: java.io.IOException:
background merge hit exception: _3:c28 _4:c1 into _5 [optimize]
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:332)
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:216)
	at com.test.search.impl.lucene.IndexManager.reindex(IndexManager.java:279)
	at com.test.search.impl.lucene.IndexManager.reindex(IndexManager.java:340)
	at
com.test.vcssearch.DefaultServiceIndexer$2.run(DefaultServiceIndexer.java:239)
	... 1 more
Caused by: java.io.IOException: background merge hit exception: _3:c28 _4:c1
into _5 [optimize]
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1787)
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1727)
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1707)
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:318)
	... 5 more
Caused by: java.io.FileNotFoundException:
/tmp/content/3615.0-3618.0/_3_1.del (No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:506)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:536)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:440)
	at org.apache.lucene.util.BitVector.<init>(BitVector.java:180)
	at org.apache.lucene.index.IndexWriter.commitMerge(IndexWriter.java:2827)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3295)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:2968)
	at
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:240)
IW 42 [Thread-86]: CMS:   no more merges pending; now return
Exception in thread "Thread-81" java.lang.RuntimeException:
com.test.search.IndexingException: java.io.FileNotFoundException:
/tmp/content/3615.0-3618.0/_3_1.del (No such file or directory)
	at
com.test.vcssearch.DefaultServiceIndexer$2.run(DefaultServiceIndexer.java:245)
	at java.lang.Thread.run(Thread.java:595)
Caused by: com.test.search.IndexingException: java.io.FileNotFoundException:
/tmp/content/3615.0-3618.0/_3_1.del (No such file or directory)
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:332)
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:216)
	at com.test.search.impl.lucene.IndexManager.reindex(IndexManager.java:279)
	at com.test.search.impl.lucene.IndexManager.reindex(IndexManager.java:340)
	at
com.test.vcssearch.DefaultServiceIndexer$2.run(DefaultServiceIndexer.java:239)
	... 1 more
Caused by: java.io.FileNotFoundException:
/tmp/content/3615.0-3618.0/_3_1.del (No such file or directory)
	at java.io.RandomAccessFile.open(Native Method)
	at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:506)
	at
org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:536)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:445)
	at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:440)
	at org.apache.lucene.util.BitVector.<init>(BitVector.java:180)
	at
org.apache.lucene.index.SegmentReader.loadDeletedDocs(SegmentReader.java:352)
	at org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:319)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:262)
	at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:205)
	at org.apache.lucene.index.IndexWriter.applyDeletes(IndexWriter.java:3441)
	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:2638)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2523)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2509)
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1744)
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1727)
	at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1707)
	at
com.test.search.impl.lucene.IndexManager.removeDocuments(IndexManager.java:318)
	... 5 more

-- 
View this message in context: http://www.nabble.com/FileNotFoundException-during-indexing-tp18766343p18775319.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message