lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Lucene 2.3.1 IndexReader and IndexWriter Synchronization Issues
Date Sat, 29 Mar 2008 07:13:51 GMT

That use case of Lucene should be fine, ie no further synchronization  
should be necessary.

Your debug output below is great, but it doesn't seem to cover the  
occurrence of that exception.  Can you post the full debug output?

Can you describe how you reindex in more detail?  Are you manually  
removing files from the index and then opening a new IndexWriter with  
create=true?

Mike

Jamie wrote:
> HI There
>
> I keep getting the following error when simultaneously reindexing  
> my documents and searching through the index.
>
> java.io.IOException: Cannot overwrite: C:\index9121\_2.cfs
>    at org.apache.lucene.store.FSDirectory.createOutput 
> (FSDirectory.java:433)
>    at org.apache.lucene.index.CompoundFileWriter.close 
> (CompoundFileWriter.java:150)
>    at org.apache.lucene.index.DocumentsWriter.createCompoundFile 
> (DocumentsWriter.java:569)
>    at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java: 
> 2565)
>    at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2397)
>    at org.apache.lucene.index.IndexWriter.closeInternal 
> (IndexWriter.java:1204)
>    at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1178)
>    at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1153)
>
> Consequently, often documents are missing from the index. Does one  
> have to perform any synchronized across IndexWriter and  
> IndexReader? I assumed that Lucene took care of this. I sincerely  
> appreciate any guidance on offer!'
>
> Many thanks
> Jamie
>
> PS: Here is the Lucene indexing debug information.
>
> IFD [http-8090-2]: setInfoStream  
> deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolic 
> y@1ae3b4d
> IW 27 [http-8090-2]: setInfoStream:  
> dir=org.apache.lucene.store.FSDirectory@C:\index9121  
> autoCommit=false  
> mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@1b7682d  
> mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@1873a6 
>  ramBufferSizeMB=16.0 maxBuffereDocs=-1 maxBuffereDeleteTerms=-1  
> maxFieldLength=50000 index=_3:c113 _4:c70 _5:c44 _6:c68
> IW 27 [Timer-0]: now flush at close
> IW 27 [Timer-0]:   flush: segment=_7 docStoreSegment=_7  
> docStoreOffset=0 flushDocs=true flushDeletes=false  
> flushDocStores=true numDocs=27 numBufDelTerms=0
> IW 27 [Timer-0]:   index before flush _3:c113 _4:c70 _5:c44 _6:c68
>
> flush postings as segment _7 numDocs=27
>
> closeDocStore: 2 files to flush to segment _7 numDocs=27
>  oldRAMSize=212992 newFlushedSize=32784 docs/MB=863.578 new/ 
> old=15.392%
> IFD [Timer-0]: now checkpoint "segments_5" [5 segments ; isCommit =  
> false]
> IFD [Timer-0]: delete pending file _9s.cfs
> IFD [Timer-0]: delete "_9s.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9y.cfs
> IFD [Timer-0]: delete "_9y.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfs
> IFD [Timer-0]: delete "_9x.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9v.cfs
> IFD [Timer-0]: delete "_9v.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9z.cfs
> IFD [Timer-0]: delete "_9z.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfs
> IFD [Timer-0]: delete "_9t.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfx
> IFD [Timer-0]: delete "_9x.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9u.cfs
> IFD [Timer-0]: delete "_9u.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9w.cfs
> IFD [Timer-0]: delete "_9w.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfx
> IFD [Timer-0]: delete "_9t.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
> try later.
> IFD [Timer-0]: now checkpoint "segments_5" [5 segments ; isCommit =  
> false]
> IFD [Timer-0]: delete pending file _9s.cfs
> IFD [Timer-0]: delete "_9s.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9y.cfs
> IFD [Timer-0]: delete "_9y.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfs
> IFD [Timer-0]: delete "_9x.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9v.cfs
> IFD [Timer-0]: delete "_9v.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9z.cfs
> IFD [Timer-0]: delete "_9z.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfs
> IFD [Timer-0]: delete "_9t.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfx
> IFD [Timer-0]: delete "_9x.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9u.cfs
> IFD [Timer-0]: delete "_9u.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9w.cfs
> IFD [Timer-0]: delete "_9w.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfx
> IFD [Timer-0]: delete "_9t.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
> try later.
> IFD [Timer-0]: delete "_7.fnm"
> IFD [Timer-0]: delete "_7.frq"
> IFD [Timer-0]: delete "_7.prx"
> IFD [Timer-0]: delete "_7.tis"
> IFD [Timer-0]: delete "_7.tii"
> IFD [Timer-0]: delete "_7.nrm"
> IFD [Timer-0]: delete "_7.fdx"
> IFD [Timer-0]: delete "_7.fdt"
> IW 27 [Timer-0]: LMP: findMerges: 5 segments
> IW 27 [Timer-0]: LMP:   level -1.0 to 5.250432: 5 segments
> IW 27 [Timer-0]: CMS: now merge
> IW 27 [Timer-0]: CMS:   index: _3:c113 _4:c70 _5:c44 _6:c68 _7:c27
> IW 27 [Timer-0]: CMS:   no more merges pending; now return
> IW 27 [Timer-0]: close: wrote segments file "segments_6"
> IFD [Timer-0]: now checkpoint "segments_6" [5 segments ; isCommit =  
> true]
> IFD [Timer-0]: delete pending file _9s.cfs
> IFD [Timer-0]: delete "_9s.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9y.cfs
> IFD [Timer-0]: delete "_9y.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfs
> IFD [Timer-0]: delete "_9x.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9v.cfs
> IFD [Timer-0]: delete "_9v.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9z.cfs
> IFD [Timer-0]: delete "_9z.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfs
> IFD [Timer-0]: delete "_9t.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfx
> IFD [Timer-0]: delete "_9x.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9u.cfs
> IFD [Timer-0]: delete "_9u.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9w.cfs
> IFD [Timer-0]: delete "_9w.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfx
> IFD [Timer-0]: delete "_9t.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfx; Will re- 
> try later.
> IFD [Timer-0]: deleteCommits: now remove commit "segments_5"
> IFD [Timer-0]: delete "segments_5"
> IW 27 [Timer-0]: at close: _3:c113 _4:c70 _5:c44 _6:c68 _7:c27
> IFD [Timer-0]: delete pending file _9s.cfs
> IFD [Timer-0]: delete "_9s.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9s.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9s.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9y.cfs
> IFD [Timer-0]: delete "_9y.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9y.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9y.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfs
> IFD [Timer-0]: delete "_9x.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9v.cfs
> IFD [Timer-0]: delete "_9v.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9v.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9v.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9z.cfs
> IFD [Timer-0]: delete "_9z.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9z.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9z.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfs
> IFD [Timer-0]: delete "_9t.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9t.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9x.cfx
> IFD [Timer-0]: delete "_9x.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9x.cfx":  
> java.io.IOException: Cannot delete C:\index9121\_9x.cfx; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9u.cfs
> IFD [Timer-0]: delete "_9u.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9u.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9u.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9w.cfs
> IFD [Timer-0]: delete "_9w.cfs"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9w.cfs":  
> java.io.IOException: Cannot delete C:\index9121\_9w.cfs; Will re- 
> try later.
> IFD [Timer-0]: delete pending file _9t.cfx
> IFD [Timer-0]: delete "_9t.cfx"
> IFD [Timer-0]: IndexFileDeleter: unable to remove file "_9t.cfx": jav


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message