lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexWriter.getReader.getVersion behavior
Date Fri, 26 Feb 2010 17:40:20 GMT
OK -- I can now see what happened.

There was a merge still running, when you called IW.commit (Lucene
Merge Thread #0).  Because IW.commit does not wait for BG merges to
finish, but IW.close does (by default), this means you'll pick up an
extra version whenever a merge is running when you call close.

Mike

On Thu, Feb 25, 2010 at 2:52 PM, Peter Keegan <peterlkeegan@gmail.com> wrote:
> I'm pretty sure this output occurred when the version number skipped +1.
> The line containing '============'. separates the close/open IndexWriter.
>
> IFD [Indexer]: setInfoStream
> deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@646f9dd9
> IW 9 [Indexer]: setInfoStream:
> dir=org.apache.lucene.store.SimpleFSDirectory@<obfuscated
> pathname>\lresumes1.search.main.1 autoCommit=false
> mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@5be44512mergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@6772cfdframBufferSizeMB=16.0
> maxBufferedDocs=-1 maxBuffereDeleteTerms=-1
> maxFieldLength=100000 index=_a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d
> _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> IW 9 [Indexer]: flush at getReader
> IW 9 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> IW 9 [UpdWriterBuild : 9]: DW:   RAM: now flush @ usedMB=15.816
> allocMB=15.816 deletesMB=0.203 triggerMB=16
> IW 9 [UpdWriterBuild : 9]:   flush: segment=_j docStoreSegment=_j
> docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false
> numDocs=1456 numBufDelTerms=1456
> IW 9 [UpdWriterBuild : 9]:   index before flush _a:C9780 _b:C1204->_b
> _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> IW 9 [UpdWriterBuild : 9]: DW: flush postings as segment _j numDocs=1456
> IW 9 [UpdWriterBuild : 9]: DW:   oldRAMSize=16584704 newFlushedSize=4969789
> docs/MB=307.202 new/old=29.966%
> IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ;
> isCommit = false]
> IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ;
> isCommit = false]
> IW 9 [UpdWriterBuild : 9]: LMP: findMerges: 10 segments
> IW 9 [UpdWriterBuild : 9]: LMP:   level 6.863048 to 7.613048: 1 segments
> IW 9 [UpdWriterBuild : 9]: LMP:   level 6.2247195 to 6.696363: 9 segments
> IW 9 [UpdWriterBuild : 9]: CMS: now merge
> IW 9 [UpdWriterBuild : 9]: CMS:   index: _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> IW 9 [UpdWriterBuild : 9]: CMS:   no more merges pending; now return
> IW 9 [Indexer]: prepareCommit: flush
> IW 9 [Indexer]:   flush: segment=_k docStoreSegment=_j docStoreOffset=1456
> flushDocs=true flushDeletes=true flushDocStores=true numDocs=509
> numBufDelTerms=509
> IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> IW 9 [Indexer]:   flush shared docStore segment _j
> IW 9 [Indexer]: DW: closeDocStore: 2 files to flush to segment _j
> numDocs=1965
> IW 9 [Indexer]: DW: flush postings as segment _k numDocs=509
> IW 9 [Indexer]: DW:   oldRAMSize=7483392 newFlushedSize=1854970
> docs/MB=287.727 new/old=24.788%
> IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit = false]
> IW 9 [Indexer]: DW: apply 1965 buffered deleted terms and 0 deleted docIDs
> and 0 deleted queries on 11 segments.
> IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit = false]
> IW 9 [Indexer]: LMP: findMerges: 11 segments
> IW 9 [Indexer]: LMP:   level 6.863048 to 7.613048: 1 segments
> IW 9 [Indexer]: LMP:   level 6.2247195 to 6.696363: 10 segments
> IW 9 [Indexer]: LMP:     1 to 11: add this merge
> IW 9 [Indexer]: add merge to pendingMerges: _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j [total 1 pending]
> IW 9 [Indexer]: CMS: now merge
> IW 9 [Indexer]: CMS:   index: _a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d
> _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j
> _k:C509->_j
> IW 9 [Indexer]: CMS:   consider merge _b:C1204->_b _c:C717->_b _d:C1220->_d
> _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j
> _k:C509->_j into _l [mergeDocStores]
> IW 9 [Indexer]: CMS:     launch new thread [Lucene Merge Thread #0]
> IW 9 [Indexer]: CMS:   no more merges pending; now return
> IW 9 [Indexer]: startCommit(): start sizeInBytes=0
> IW 9 [Indexer]: startCommit index=_a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j changeCount=7
> IW 9 [Lucene Merge Thread #0]: CMS:   merge thread: start
> IW 9 [Indexer]: now sync _k.fnm
> IW 9 [Lucene Merge Thread #0]: now merge
>  merge=_b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f
> _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j into _l
> [mergeDocStores]
>  merge=org.apache.lucene.index.MergePolicy$OneMerge@5203ef34
>  index=_a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d
> _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j
> IW 9 [Lucene Merge Thread #0]: merging _b:C1204->_b _c:C717->_b _d:C1220->_d
> _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j
> _k:C509->_j into _l [mergeDocStores]
> IW 9 [Indexer]: now sync _k.tii
> IW 9 [Indexer]: now sync _k.tis
> IW 9 [Lucene Merge Thread #0]: merge: total 9909 docs
> IW 9 [Indexer]: now sync _j.tis
> IW 9 [Indexer]: now sync _j.tii
> IW 9 [Indexer]: now sync _j.fdx
> IW 9 [Indexer]: now sync _j.fdt
> IW 9 [Indexer]: now sync _k.prx
> IW 9 [Indexer]: now sync _k.nrm
> IW 9 [Indexer]: now sync _j.nrm
> IW 9 [Indexer]: now sync _j.prx
> IW 9 [Indexer]: now sync _j.fnm
> IW 9 [Indexer]: now sync _k.frq
> IW 9 [Indexer]: now sync _j.frq
> IW 9 [Indexer]: done all syncs
> IW 9 [Indexer]: flush at getReader
> IW 9 [Indexer]:   flush: segment=null docStoreSegment=_k docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=true numDocs=0
> numBufDelTerms=0
> IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j
> IW 9 [Indexer]:   flush shared docStore segment _k
> IW 9 [Indexer]: DW: closeDocStore: 0 files to flush to segment _k numDocs=0
> IW 9 [Indexer]: commit: start
> IW 9 [Indexer]: commit: already prepared
> IW 9 [Indexer]: commit: pendingCommit != null
> IW 9 [Indexer]: commit: wrote segments file "segments_c"
> IFD [Indexer]: now checkpoint "segments_c" [11 segments ; isCommit = true]
> IFD [Indexer]: deleteCommits: now decRef commit "segments_b"
> IFD [Indexer]: delete "segments_b"
> IW 9 [Indexer]: commit: done
> IW 9 [Indexer]: flush at getReader
> IW 9 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j
> IW 9 [Indexer]: now flush at close
> IW 9 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j
> IW 9 [Indexer]: CMS: now merge
> IW 9 [Indexer]: CMS:   index: _a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d
> _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j
> _k:C509->_j
> IW 9 [Indexer]: CMS:   no more merges pending; now return
> IW 9 [Lucene Merge Thread #0]: commitMerge: _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j into _l [mergeDocStores] index=_a:C9780
> _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f
> _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j
> IW 9 [Lucene Merge Thread #0]: commitMergeDeletes _b:C1204->_b _c:C717->_b
> _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j _k:C509->_j into _l [mergeDocStores]
> IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ;
> isCommit = false]
> IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ;
> isCommit = false]
> IW 9 [Lucene Merge Thread #0]: CMS:   merge thread: done
> IW 9 [Indexer]: now call final commit()
> IW 9 [Indexer]: startCommit(): start sizeInBytes=0
> IW 9 [Indexer]: startCommit index=_a:C9780 _l:C9909 changeCount=8
> IW 9 [Indexer]: now sync _l.nrm
> IW 9 [Indexer]: now sync _l.fnm
> IW 9 [Indexer]: now sync _l.prx
> IW 9 [Indexer]: now sync _l.frq
> IW 9 [Indexer]: now sync _l.tis
> IW 9 [Indexer]: now sync _l.fdx
> IW 9 [Indexer]: now sync _l.fdt
> IW 9 [Indexer]: now sync _l.tii
> IW 9 [Indexer]: done all syncs
> IW 9 [Indexer]: commit: pendingCommit != null
> IW 9 [Indexer]: commit: wrote segments file "segments_d"
> IFD [Indexer]: now checkpoint "segments_d" [2 segments ; isCommit = true]
> IFD [Indexer]: deleteCommits: now decRef commit "segments_c"
> IFD [Indexer]: delete "_h.prx"
> IFD [Indexer]: delete "_f.tis"
> IFD [Indexer]: delete "_g.tis"
> IFD [Indexer]: delete "_d.tii"
> IFD [Indexer]: delete "_e.frq"
> IFD [Indexer]: delete "_k.fnm"
> IFD [Indexer]: delete "_c.tii"
> IFD [Indexer]: delete "_h.nrm"
> IFD [Indexer]: delete "_i.tis"
> IFD [Indexer]: delete "_k.tii"
> IFD [Indexer]: delete "_h.fdt"
> IFD [Indexer]: delete "_c.tis"
> IFD [Indexer]: delete "_g.tii"
> IFD [Indexer]: delete "_h.fdx"
> IFD [Indexer]: delete "_c.nrm"
> IFD [Indexer]: delete "_i.tii"
> IFD [Indexer]: delete "_d.tis"
> IFD [Indexer]: delete "_f.tii"
> IFD [Indexer]: delete "_e.fnm"
> IFD [Indexer]: delete "_k.tis"
> IFD [Indexer]: delete "_d.frq"
> IFD [Indexer]: delete "_f.nrm"
> IFD [Indexer]: delete "_h.fnm"
> IFD [Indexer]: delete "segments_c"
> IFD [Indexer]: delete "_c.prx"
> IFD [Indexer]: delete "_d.fnm"
> IFD [Indexer]: delete "_h.frq"
> IFD [Indexer]: delete "_h.tis"
> IFD [Indexer]: delete "_f.fdt"
> IFD [Indexer]: delete "_j.tis"
> IFD [Indexer]: delete "_f.fdx"
> IFD [Indexer]: delete "_g.nrm"
> IFD [Indexer]: delete "_f.prx"
> IFD [Indexer]: delete "_j.tii"
> IFD [Indexer]: delete "_h.tii"
> IFD [Indexer]: delete "_e.prx"
> IFD [Indexer]: delete "_c.frq"
> IFD [Indexer]: delete "_i.prx"
> IFD [Indexer]: delete "_j.fdx"
> IFD [Indexer]: delete "_i.nrm"
> IFD [Indexer]: delete "_j.fdt"
> IFD [Indexer]: delete "_k.prx"
> IFD [Indexer]: delete "_b.nrm"
> IFD [Indexer]: delete "_e.tis"
> IFD [Indexer]: delete "_e.tii"
> IFD [Indexer]: delete "_g.fnm"
> IFD [Indexer]: delete "_k.nrm"
> IFD [Indexer]: delete "_j.nrm"
> IFD [Indexer]: delete "_j.prx"
> IFD [Indexer]: delete "_e.nrm"
> IFD [Indexer]: delete "_g.prx"
> IFD [Indexer]: delete "_b.prx"
> IFD [Indexer]: delete "_i.frq"
> IFD [Indexer]: delete "_b.tis"
> IFD [Indexer]: delete "_d.prx"
> IFD [Indexer]: delete "_g.frq"
> IFD [Indexer]: delete "_d.fdx"
> IFD [Indexer]: delete "_b.tii"
> IFD [Indexer]: delete "_b.frq"
> IFD [Indexer]: delete "_d.fdt"
> IFD [Indexer]: delete "_f.fnm"
> IFD [Indexer]: delete "_j.fnm"
> IFD [Indexer]: delete "_b.fnm"
> IFD [Indexer]: delete "_d.nrm"
> IFD [Indexer]: delete "_b.fdt"
> IFD [Indexer]: delete "_k.frq"
> IFD [Indexer]: delete "_c.fnm"
> IFD [Indexer]: delete "_j.frq"
> IFD [Indexer]: delete "_i.fnm"
> IFD [Indexer]: delete "_f.frq"
> IFD [Indexer]: delete "_b.fdx"
> IW 9 [Indexer]: commit: done
> IW 9 [Indexer]: at close: _a:C9780 _l:C9909
>
> =====================================================
>
> IFD [Indexer]: setInfoStream
> deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@68d349c1
> IW 10 [Indexer]: setInfoStream:
> dir=org.apache.lucene.store.SimpleFSDirectory@<obfuscated
> pathname>\lresumes1.search.main.1 autoCommit=false
> mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@78f53abmergeScheduler=org.apache.lucene.index.ConcurrentMergeScheduler@52be036dramBufferSizeMB=16.0
> maxBufferedDocs=-1 maxBuffereDeleteTerms=-1
> maxFieldLength=100000 index=_a:C9780 _l:C9909
> IW 10 [Indexer]: flush at getReader
> IW 10 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909
> IW 10 [UpdWriterBuild : 10]: DW:   RAM: now flush @ usedMB=15.789
> allocMB=15.789 deletesMB=0.212 triggerMB=16
> IW 10 [UpdWriterBuild : 10]:   flush: segment=_m docStoreSegment=_m
> docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false
> numDocs=1526 numBufDelTerms=1526
> IW 10 [UpdWriterBuild : 10]:   index before flush _a:C9780 _l:C9909
> IW 10 [UpdWriterBuild : 10]: DW: flush postings as segment _m numDocs=1526
> IW 10 [UpdWriterBuild : 10]: DW:   oldRAMSize=16556032
> newFlushedSize=4788823 docs/MB=334.138 new/old=28.925%
> IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ;
> isCommit = false]
> IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ;
> isCommit = false]
> IW 10 [UpdWriterBuild : 10]: LMP: findMerges: 3 segments
> IW 10 [UpdWriterBuild : 10]: LMP:   level 6.8642592 to 7.6142592: 2 segments
> IW 10 [UpdWriterBuild : 10]: LMP:   level 6.2247195 to 6.6802545: 1 segments
> IW 10 [UpdWriterBuild : 10]: CMS: now merge
> IW 10 [UpdWriterBuild : 10]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m
> IW 10 [UpdWriterBuild : 10]: CMS:   no more merges pending; now return
> IW 10 [Indexer]: prepareCommit: flush
> IW 10 [Indexer]:   flush: segment=_n docStoreSegment=_m docStoreOffset=1526
> flushDocs=true flushDeletes=true flushDocStores=true numDocs=501
> numBufDelTerms=501
> IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> IW 10 [Indexer]:   flush shared docStore segment _m
> IW 10 [Indexer]: DW: closeDocStore: 2 files to flush to segment _m
> numDocs=2027
> IW 10 [Indexer]: DW: flush postings as segment _n numDocs=501
> IW 10 [Indexer]: DW:   oldRAMSize=6927360 newFlushedSize=1659342
> docs/MB=316.593 new/old=23.953%
> IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit = false]
> IW 10 [Indexer]: DW: apply 2027 buffered deleted terms and 0 deleted docIDs
> and 0 deleted queries on 4 segments.
> IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit = false]
> IW 10 [Indexer]: LMP: findMerges: 4 segments
> IW 10 [Indexer]: LMP:   level 6.8642592 to 7.6142592: 2 segments
> IW 10 [Indexer]: LMP:   level 6.2247195 to 6.6802545: 1 segments
> IW 10 [Indexer]: LMP:   level -1.0 to 6.220011: 1 segments
> IW 10 [Indexer]: CMS: now merge
> IW 10 [Indexer]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
> IW 10 [Indexer]: CMS:   no more merges pending; now return
> IW 10 [Indexer]: startCommit(): start sizeInBytes=0
> IW 10 [Indexer]: startCommit index=_a:C9780 _l:C9909 _m:C1526->_m
> _n:C501->_m changeCount=6
> IW 10 [Indexer]: now sync _m.frq
> IW 10 [Indexer]: now sync _m.prx
> IW 10 [Indexer]: now sync _n.tis
> IW 10 [Indexer]: now sync _m.tis
> IW 10 [Indexer]: now sync _n.frq
> IW 10 [Indexer]: now sync _n.tii
> IW 10 [Indexer]: now sync _m.tii
> IW 10 [Indexer]: now sync _m.fnm
> IW 10 [Indexer]: now sync _n.fnm
> IW 10 [Indexer]: now sync _n.nrm
> IW 10 [Indexer]: now sync _n.prx
> IW 10 [Indexer]: now sync _m.nrm
> IW 10 [Indexer]: now sync _m.fdt
> IW 10 [Indexer]: now sync _m.fdx
> IW 10 [Indexer]: done all syncs
> IW 10 [Indexer]: flush at getReader
> IW 10 [Indexer]:   flush: segment=null docStoreSegment=_n docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=true numDocs=0
> numBufDelTerms=0
> IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> _n:C501->_m
> IW 10 [Indexer]:   flush shared docStore segment _n
> IW 10 [Indexer]: DW: closeDocStore: 0 files to flush to segment _n numDocs=0
> IW 10 [Indexer]: commit: start
> IW 10 [Indexer]: commit: already prepared
> IW 10 [Indexer]: commit: pendingCommit != null
> IW 10 [Indexer]: commit: wrote segments file "segments_e"
> IFD [Indexer]: now checkpoint "segments_e" [4 segments ; isCommit = true]
> IFD [Indexer]: deleteCommits: now decRef commit "segments_d"
> IFD [Indexer]: delete "segments_d"
> IW 10 [Indexer]: commit: done
> IW 10 [Indexer]: flush at getReader
> IW 10 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> _n:C501->_m
> IW 10 [Indexer]: now flush at close
> IW 10 [Indexer]:   flush: segment=null docStoreSegment=null docStoreOffset=0
> flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> numBufDelTerms=0
> IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> _n:C501->_m
> IW 10 [Indexer]: CMS: now merge
> IW 10 [Indexer]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
> IW 10 [Indexer]: CMS:   no more merges pending; now return
> IW 10 [Indexer]: now call final commit()
> IW 10 [Indexer]: startCommit(): start sizeInBytes=0
> IW 10 [Indexer]:   skip startCommit(): no changes pending
> IW 10 [Indexer]: commit: pendingCommit == null; skip
> IW 10 [Indexer]: commit: done
> IW 10 [Indexer]: at close: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
>
>
> Peter
>
> On Thu, Feb 25, 2010 at 1:13 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>> Do you know the place in the infoStream output where you got a reader
>> with the wrong (unexplained extra +1) version?  If so, can you post
>> the infoStream output up to that point?
>>
>> Mike
>>
>> On Thu, Feb 25, 2010 at 10:22 AM, Peter Keegan <peterlkeegan@gmail.com>
>> wrote:
>> > I've reproduced this and I have a bunch of infoStream log files. Since
>> the
>> > messages have no timestamps, it's hard to tell where the relevant entries
>> > are. What should I be looking for?
>> >
>> > Peter
>> >
>> > On Mon, Feb 22, 2010 at 3:58 PM, Peter Keegan <peterlkeegan@gmail.com
>> >wrote:
>> >
>> >> I'm pretty sure there are flushes and segment merges going on, but as
>> you
>> >> said, that shouldn't affect the version increment. I'll see what I can
>> do to
>> >> get infoStream output.
>> >>
>> >> Thanks,
>> >> Peter
>> >>
>> >>
>> >> On Mon, Feb 22, 2010 at 2:30 PM, Michael McCandless <
>> >> lucene@mikemccandless.com> wrote:
>> >>
>> >>> Well I'm at a loss then.  The version should only increment on commit.
>> >>>
>> >>> Can you make it all happen when infoStream is on, and post back?
>> >>>
>> >>> Mike
>> >>>
>> >>> On Mon, Feb 22, 2010 at 12:35 PM, Peter Keegan <peterlkeegan@gmail.com
>> >
>> >>> wrote:
>> >>> > Only one writer thread and one writer process.
>> >>> > I'm calling IndexWriter(Directory d, Analyzer a, boolean create,
>> >>> > MaxFieldLength mfl), which sets autocommit=false.
>> >>> >
>> >>> > Peter
>> >>> >
>> >>> > On Mon, Feb 22, 2010 at 12:24 PM, Michael McCandless <
>> >>> > lucene@mikemccandless.com> wrote:
>> >>> >
>> >>> >> That's curious.
>> >>> >>
>> >>> >> It's only on prepareCommit (or, commit, if you didn't first
prepare,
>> >>> >> since that will call prepareCommit internally) that this version
>> >>> >> should increase.
>> >>> >>
>> >>> >> Is there only 1 thread doing this?
>> >>> >>
>> >>> >> Oh, and, are you passing false for autoCommit?
>> >>> >>
>> >>> >> Mike
>> >>> >>
>> >>> >> On Mon, Feb 22, 2010 at 11:43 AM, Peter Keegan <
>> peterlkeegan@gmail.com
>> >>> >
>> >>> >> wrote:
>> >>> >> > Using Lucene 2.9.1, I have the following pseudocode which
gets
>> >>> repeated
>> >>> >> at
>> >>> >> > regular intervals:
>> >>> >> >
>> >>> >> > 1. FSDirectory dir = FSDirectory.open(java.io.File);
>> >>> >> > 2. dir.setLockFactory(new SingleInstanceLockFactory());
>> >>> >> > 3. IndexWriter writer = new IndexWriter(dir, Analyzer,
false,
>> >>> >> maxFieldLen)
>> >>> >> > 4. writer.getReader().getVersion();
>> >>> >> > 5. writer.prepareCommit();
>> >>> >> > 6. writer.getReader().getVersion();
>> >>> >> > 7. writer.commit();
>> >>> >> > 8. writer.close();
>> >>> >> >
>> >>> >> > I'm using the version number to keep external data in
synch with
>> the
>> >>> >> index.
>> >>> >> > Usually, the version number from (6) is 1 greater than
from (4)
>> and
>> >>> the
>> >>> >> > version from (4) equals the version from the previous
(6). At
>> least
>> >>> once
>> >>> >> a
>> >>> >> > day, however, the version from (4) is 1 greater than from
the
>> >>> previous
>> >>> >> (6).
>> >>> >> > What would explain this sporadic behavior of version numbers?
>> >>> >> >
>> >>> >> > Thanks,
>> >>> >> > Peter
>> >>> >> >
>> >>> >>
>> >>> >>
>> ---------------------------------------------------------------------
>> >>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >>> >> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>> >>
>> >>> >>
>> >>> >
>> >>>
>> >>> ---------------------------------------------------------------------
>> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> >>>
>> >>>
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message