lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Keegan <peterlkee...@gmail.com>
Subject Re: IndexWriter.getReader.getVersion behavior
Date Fri, 26 Feb 2010 19:40:56 GMT
Is there a way for the application to wait for the BG commit to finish
before it calls IW.close? If so, would this prevent the extra version? The
extra version causes the app. to think that the external data it committed
is out of synch with the index, which requires the app to do extra
processing to re-synch.

Thanks,
Peter


On Fri, Feb 26, 2010 at 12:40 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> OK -- I can now see what happened.
>
> There was a merge still running, when you called IW.commit (Lucene
> Merge Thread #0).  Because IW.commit does not wait for BG merges to
> finish, but IW.close does (by default), this means you'll pick up an
> extra version whenever a merge is running when you call close.
>
> Mike
>
> On Thu, Feb 25, 2010 at 2:52 PM, Peter Keegan <peterlkeegan@gmail.com>
> wrote:
> > I'm pretty sure this output occurred when the version number skipped +1.
> > The line containing '============'. separates the close/open IndexWriter.
> >
> > IFD [Indexer]: setInfoStream
> >
> deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@646f9dd9
> > IW 9 [Indexer]: setInfoStream:
> > dir=org.apache.lucene.store.SimpleFSDirectory@<obfuscated
> > pathname>\lresumes1.search.main.1 autoCommit=false
> >
> mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@5be44512mergeScheduler
> =org.apache.lucene.index.ConcurrentMergeScheduler@6772cfdframBufferSizeMB
> =16.0
> > maxBufferedDocs=-1 maxBuffereDeleteTerms=-1
> > maxFieldLength=100000 index=_a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d
> > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> > IW 9 [Indexer]: flush at getReader
> > IW 9 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > IW 9 [UpdWriterBuild : 9]: DW:   RAM: now flush @ usedMB=15.816
> > allocMB=15.816 deletesMB=0.203 triggerMB=16
> > IW 9 [UpdWriterBuild : 9]:   flush: segment=_j docStoreSegment=_j
> > docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false
> > numDocs=1456 numBufDelTerms=1456
> > IW 9 [UpdWriterBuild : 9]:   index before flush _a:C9780 _b:C1204->_b
> > _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f
> _h:C1291->_h
> > _i:C703->_h
> > IW 9 [UpdWriterBuild : 9]: DW: flush postings as segment _j numDocs=1456
> > IW 9 [UpdWriterBuild : 9]: DW:   oldRAMSize=16584704
> newFlushedSize=4969789
> > docs/MB=307.202 new/old=29.966%
> > IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ;
> > isCommit = false]
> > IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ;
> > isCommit = false]
> > IW 9 [UpdWriterBuild : 9]: LMP: findMerges: 10 segments
> > IW 9 [UpdWriterBuild : 9]: LMP:   level 6.863048 to 7.613048: 1 segments
> > IW 9 [UpdWriterBuild : 9]: LMP:   level 6.2247195 to 6.696363: 9 segments
> > IW 9 [UpdWriterBuild : 9]: CMS: now merge
> > IW 9 [UpdWriterBuild : 9]: CMS:   index: _a:C9780 _b:C1204->_b
> _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j
> > IW 9 [UpdWriterBuild : 9]: CMS:   no more merges pending; now return
> > IW 9 [Indexer]: prepareCommit: flush
> > IW 9 [Indexer]:   flush: segment=_k docStoreSegment=_j
> docStoreOffset=1456
> > flushDocs=true flushDeletes=true flushDocStores=true numDocs=509
> > numBufDelTerms=509
> > IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j
> > IW 9 [Indexer]:   flush shared docStore segment _j
> > IW 9 [Indexer]: DW: closeDocStore: 2 files to flush to segment _j
> > numDocs=1965
> > IW 9 [Indexer]: DW: flush postings as segment _k numDocs=509
> > IW 9 [Indexer]: DW:   oldRAMSize=7483392 newFlushedSize=1854970
> > docs/MB=287.727 new/old=24.788%
> > IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit =
> false]
> > IW 9 [Indexer]: DW: apply 1965 buffered deleted terms and 0 deleted
> docIDs
> > and 0 deleted queries on 11 segments.
> > IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit =
> false]
> > IW 9 [Indexer]: LMP: findMerges: 11 segments
> > IW 9 [Indexer]: LMP:   level 6.863048 to 7.613048: 1 segments
> > IW 9 [Indexer]: LMP:   level 6.2247195 to 6.696363: 10 segments
> > IW 9 [Indexer]: LMP:     1 to 11: add this merge
> > IW 9 [Indexer]: add merge to pendingMerges: _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j [total 1 pending]
> > IW 9 [Indexer]: CMS: now merge
> > IW 9 [Indexer]: CMS:   index: _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d
> > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> > _k:C509->_j
> > IW 9 [Indexer]: CMS:   consider merge _b:C1204->_b _c:C717->_b
> _d:C1220->_d
> > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> > _k:C509->_j into _l [mergeDocStores]
> > IW 9 [Indexer]: CMS:     launch new thread [Lucene Merge Thread #0]
> > IW 9 [Indexer]: CMS:   no more merges pending; now return
> > IW 9 [Indexer]: startCommit(): start sizeInBytes=0
> > IW 9 [Indexer]: startCommit index=_a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j changeCount=7
> > IW 9 [Lucene Merge Thread #0]: CMS:   merge thread: start
> > IW 9 [Indexer]: now sync _k.fnm
> > IW 9 [Lucene Merge Thread #0]: now merge
> >  merge=_b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f
> > _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j into
_l
> > [mergeDocStores]
> >  merge=org.apache.lucene.index.MergePolicy$OneMerge@5203ef34
> >  index=_a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d
> > _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j
> _k:C509->_j
> > IW 9 [Lucene Merge Thread #0]: merging _b:C1204->_b _c:C717->_b
> _d:C1220->_d
> > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> > _k:C509->_j into _l [mergeDocStores]
> > IW 9 [Indexer]: now sync _k.tii
> > IW 9 [Indexer]: now sync _k.tis
> > IW 9 [Lucene Merge Thread #0]: merge: total 9909 docs
> > IW 9 [Indexer]: now sync _j.tis
> > IW 9 [Indexer]: now sync _j.tii
> > IW 9 [Indexer]: now sync _j.fdx
> > IW 9 [Indexer]: now sync _j.fdt
> > IW 9 [Indexer]: now sync _k.prx
> > IW 9 [Indexer]: now sync _k.nrm
> > IW 9 [Indexer]: now sync _j.nrm
> > IW 9 [Indexer]: now sync _j.prx
> > IW 9 [Indexer]: now sync _j.fnm
> > IW 9 [Indexer]: now sync _k.frq
> > IW 9 [Indexer]: now sync _j.frq
> > IW 9 [Indexer]: done all syncs
> > IW 9 [Indexer]: flush at getReader
> > IW 9 [Indexer]:   flush: segment=null docStoreSegment=_k docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=true numDocs=0
> > numBufDelTerms=0
> > IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j
> > IW 9 [Indexer]:   flush shared docStore segment _k
> > IW 9 [Indexer]: DW: closeDocStore: 0 files to flush to segment _k
> numDocs=0
> > IW 9 [Indexer]: commit: start
> > IW 9 [Indexer]: commit: already prepared
> > IW 9 [Indexer]: commit: pendingCommit != null
> > IW 9 [Indexer]: commit: wrote segments file "segments_c"
> > IFD [Indexer]: now checkpoint "segments_c" [11 segments ; isCommit =
> true]
> > IFD [Indexer]: deleteCommits: now decRef commit "segments_b"
> > IFD [Indexer]: delete "segments_b"
> > IW 9 [Indexer]: commit: done
> > IW 9 [Indexer]: flush at getReader
> > IW 9 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j
> > IW 9 [Indexer]: now flush at close
> > IW 9 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 9 [Indexer]:   index before flush _a:C9780 _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j
> > IW 9 [Indexer]: CMS: now merge
> > IW 9 [Indexer]: CMS:   index: _a:C9780 _b:C1204->_b _c:C717->_b
> _d:C1220->_d
> > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h
> _j:C1456->_j
> > _k:C509->_j
> > IW 9 [Indexer]: CMS:   no more merges pending; now return
> > IW 9 [Lucene Merge Thread #0]: commitMerge: _b:C1204->_b _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j into _l [mergeDocStores] index=_a:C9780
> > _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f
> _g:C858->_f
> > _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j
> > IW 9 [Lucene Merge Thread #0]: commitMergeDeletes _b:C1204->_b
> _c:C717->_b
> > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h
> _i:C703->_h
> > _j:C1456->_j _k:C509->_j into _l [mergeDocStores]
> > IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ;
> > isCommit = false]
> > IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ;
> > isCommit = false]
> > IW 9 [Lucene Merge Thread #0]: CMS:   merge thread: done
> > IW 9 [Indexer]: now call final commit()
> > IW 9 [Indexer]: startCommit(): start sizeInBytes=0
> > IW 9 [Indexer]: startCommit index=_a:C9780 _l:C9909 changeCount=8
> > IW 9 [Indexer]: now sync _l.nrm
> > IW 9 [Indexer]: now sync _l.fnm
> > IW 9 [Indexer]: now sync _l.prx
> > IW 9 [Indexer]: now sync _l.frq
> > IW 9 [Indexer]: now sync _l.tis
> > IW 9 [Indexer]: now sync _l.fdx
> > IW 9 [Indexer]: now sync _l.fdt
> > IW 9 [Indexer]: now sync _l.tii
> > IW 9 [Indexer]: done all syncs
> > IW 9 [Indexer]: commit: pendingCommit != null
> > IW 9 [Indexer]: commit: wrote segments file "segments_d"
> > IFD [Indexer]: now checkpoint "segments_d" [2 segments ; isCommit = true]
> > IFD [Indexer]: deleteCommits: now decRef commit "segments_c"
> > IFD [Indexer]: delete "_h.prx"
> > IFD [Indexer]: delete "_f.tis"
> > IFD [Indexer]: delete "_g.tis"
> > IFD [Indexer]: delete "_d.tii"
> > IFD [Indexer]: delete "_e.frq"
> > IFD [Indexer]: delete "_k.fnm"
> > IFD [Indexer]: delete "_c.tii"
> > IFD [Indexer]: delete "_h.nrm"
> > IFD [Indexer]: delete "_i.tis"
> > IFD [Indexer]: delete "_k.tii"
> > IFD [Indexer]: delete "_h.fdt"
> > IFD [Indexer]: delete "_c.tis"
> > IFD [Indexer]: delete "_g.tii"
> > IFD [Indexer]: delete "_h.fdx"
> > IFD [Indexer]: delete "_c.nrm"
> > IFD [Indexer]: delete "_i.tii"
> > IFD [Indexer]: delete "_d.tis"
> > IFD [Indexer]: delete "_f.tii"
> > IFD [Indexer]: delete "_e.fnm"
> > IFD [Indexer]: delete "_k.tis"
> > IFD [Indexer]: delete "_d.frq"
> > IFD [Indexer]: delete "_f.nrm"
> > IFD [Indexer]: delete "_h.fnm"
> > IFD [Indexer]: delete "segments_c"
> > IFD [Indexer]: delete "_c.prx"
> > IFD [Indexer]: delete "_d.fnm"
> > IFD [Indexer]: delete "_h.frq"
> > IFD [Indexer]: delete "_h.tis"
> > IFD [Indexer]: delete "_f.fdt"
> > IFD [Indexer]: delete "_j.tis"
> > IFD [Indexer]: delete "_f.fdx"
> > IFD [Indexer]: delete "_g.nrm"
> > IFD [Indexer]: delete "_f.prx"
> > IFD [Indexer]: delete "_j.tii"
> > IFD [Indexer]: delete "_h.tii"
> > IFD [Indexer]: delete "_e.prx"
> > IFD [Indexer]: delete "_c.frq"
> > IFD [Indexer]: delete "_i.prx"
> > IFD [Indexer]: delete "_j.fdx"
> > IFD [Indexer]: delete "_i.nrm"
> > IFD [Indexer]: delete "_j.fdt"
> > IFD [Indexer]: delete "_k.prx"
> > IFD [Indexer]: delete "_b.nrm"
> > IFD [Indexer]: delete "_e.tis"
> > IFD [Indexer]: delete "_e.tii"
> > IFD [Indexer]: delete "_g.fnm"
> > IFD [Indexer]: delete "_k.nrm"
> > IFD [Indexer]: delete "_j.nrm"
> > IFD [Indexer]: delete "_j.prx"
> > IFD [Indexer]: delete "_e.nrm"
> > IFD [Indexer]: delete "_g.prx"
> > IFD [Indexer]: delete "_b.prx"
> > IFD [Indexer]: delete "_i.frq"
> > IFD [Indexer]: delete "_b.tis"
> > IFD [Indexer]: delete "_d.prx"
> > IFD [Indexer]: delete "_g.frq"
> > IFD [Indexer]: delete "_d.fdx"
> > IFD [Indexer]: delete "_b.tii"
> > IFD [Indexer]: delete "_b.frq"
> > IFD [Indexer]: delete "_d.fdt"
> > IFD [Indexer]: delete "_f.fnm"
> > IFD [Indexer]: delete "_j.fnm"
> > IFD [Indexer]: delete "_b.fnm"
> > IFD [Indexer]: delete "_d.nrm"
> > IFD [Indexer]: delete "_b.fdt"
> > IFD [Indexer]: delete "_k.frq"
> > IFD [Indexer]: delete "_c.fnm"
> > IFD [Indexer]: delete "_j.frq"
> > IFD [Indexer]: delete "_i.fnm"
> > IFD [Indexer]: delete "_f.frq"
> > IFD [Indexer]: delete "_b.fdx"
> > IW 9 [Indexer]: commit: done
> > IW 9 [Indexer]: at close: _a:C9780 _l:C9909
> >
> > =====================================================
> >
> > IFD [Indexer]: setInfoStream
> >
> deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@68d349c1
> > IW 10 [Indexer]: setInfoStream:
> > dir=org.apache.lucene.store.SimpleFSDirectory@<obfuscated
> > pathname>\lresumes1.search.main.1 autoCommit=false
> >
> mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@78f53abmergeScheduler
> =org.apache.lucene.index.ConcurrentMergeScheduler@52be036dramBufferSizeMB
> =16.0
> > maxBufferedDocs=-1 maxBuffereDeleteTerms=-1
> > maxFieldLength=100000 index=_a:C9780 _l:C9909
> > IW 10 [Indexer]: flush at getReader
> > IW 10 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909
> > IW 10 [UpdWriterBuild : 10]: DW:   RAM: now flush @ usedMB=15.789
> > allocMB=15.789 deletesMB=0.212 triggerMB=16
> > IW 10 [UpdWriterBuild : 10]:   flush: segment=_m docStoreSegment=_m
> > docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false
> > numDocs=1526 numBufDelTerms=1526
> > IW 10 [UpdWriterBuild : 10]:   index before flush _a:C9780 _l:C9909
> > IW 10 [UpdWriterBuild : 10]: DW: flush postings as segment _m
> numDocs=1526
> > IW 10 [UpdWriterBuild : 10]: DW:   oldRAMSize=16556032
> > newFlushedSize=4788823 docs/MB=334.138 new/old=28.925%
> > IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ;
> > isCommit = false]
> > IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ;
> > isCommit = false]
> > IW 10 [UpdWriterBuild : 10]: LMP: findMerges: 3 segments
> > IW 10 [UpdWriterBuild : 10]: LMP:   level 6.8642592 to 7.6142592: 2
> segments
> > IW 10 [UpdWriterBuild : 10]: LMP:   level 6.2247195 to 6.6802545: 1
> segments
> > IW 10 [UpdWriterBuild : 10]: CMS: now merge
> > IW 10 [UpdWriterBuild : 10]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m
> > IW 10 [UpdWriterBuild : 10]: CMS:   no more merges pending; now return
> > IW 10 [Indexer]: prepareCommit: flush
> > IW 10 [Indexer]:   flush: segment=_n docStoreSegment=_m
> docStoreOffset=1526
> > flushDocs=true flushDeletes=true flushDocStores=true numDocs=501
> > numBufDelTerms=501
> > IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> > IW 10 [Indexer]:   flush shared docStore segment _m
> > IW 10 [Indexer]: DW: closeDocStore: 2 files to flush to segment _m
> > numDocs=2027
> > IW 10 [Indexer]: DW: flush postings as segment _n numDocs=501
> > IW 10 [Indexer]: DW:   oldRAMSize=6927360 newFlushedSize=1659342
> > docs/MB=316.593 new/old=23.953%
> > IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit =
> false]
> > IW 10 [Indexer]: DW: apply 2027 buffered deleted terms and 0 deleted
> docIDs
> > and 0 deleted queries on 4 segments.
> > IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit =
> false]
> > IW 10 [Indexer]: LMP: findMerges: 4 segments
> > IW 10 [Indexer]: LMP:   level 6.8642592 to 7.6142592: 2 segments
> > IW 10 [Indexer]: LMP:   level 6.2247195 to 6.6802545: 1 segments
> > IW 10 [Indexer]: LMP:   level -1.0 to 6.220011: 1 segments
> > IW 10 [Indexer]: CMS: now merge
> > IW 10 [Indexer]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
> > IW 10 [Indexer]: CMS:   no more merges pending; now return
> > IW 10 [Indexer]: startCommit(): start sizeInBytes=0
> > IW 10 [Indexer]: startCommit index=_a:C9780 _l:C9909 _m:C1526->_m
> > _n:C501->_m changeCount=6
> > IW 10 [Indexer]: now sync _m.frq
> > IW 10 [Indexer]: now sync _m.prx
> > IW 10 [Indexer]: now sync _n.tis
> > IW 10 [Indexer]: now sync _m.tis
> > IW 10 [Indexer]: now sync _n.frq
> > IW 10 [Indexer]: now sync _n.tii
> > IW 10 [Indexer]: now sync _m.tii
> > IW 10 [Indexer]: now sync _m.fnm
> > IW 10 [Indexer]: now sync _n.fnm
> > IW 10 [Indexer]: now sync _n.nrm
> > IW 10 [Indexer]: now sync _n.prx
> > IW 10 [Indexer]: now sync _m.nrm
> > IW 10 [Indexer]: now sync _m.fdt
> > IW 10 [Indexer]: now sync _m.fdx
> > IW 10 [Indexer]: done all syncs
> > IW 10 [Indexer]: flush at getReader
> > IW 10 [Indexer]:   flush: segment=null docStoreSegment=_n
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=true numDocs=0
> > numBufDelTerms=0
> > IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> > _n:C501->_m
> > IW 10 [Indexer]:   flush shared docStore segment _n
> > IW 10 [Indexer]: DW: closeDocStore: 0 files to flush to segment _n
> numDocs=0
> > IW 10 [Indexer]: commit: start
> > IW 10 [Indexer]: commit: already prepared
> > IW 10 [Indexer]: commit: pendingCommit != null
> > IW 10 [Indexer]: commit: wrote segments file "segments_e"
> > IFD [Indexer]: now checkpoint "segments_e" [4 segments ; isCommit = true]
> > IFD [Indexer]: deleteCommits: now decRef commit "segments_d"
> > IFD [Indexer]: delete "segments_d"
> > IW 10 [Indexer]: commit: done
> > IW 10 [Indexer]: flush at getReader
> > IW 10 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> > _n:C501->_m
> > IW 10 [Indexer]: now flush at close
> > IW 10 [Indexer]:   flush: segment=null docStoreSegment=null
> docStoreOffset=0
> > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0
> > numBufDelTerms=0
> > IW 10 [Indexer]:   index before flush _a:C9780 _l:C9909 _m:C1526->_m
> > _n:C501->_m
> > IW 10 [Indexer]: CMS: now merge
> > IW 10 [Indexer]: CMS:   index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
> > IW 10 [Indexer]: CMS:   no more merges pending; now return
> > IW 10 [Indexer]: now call final commit()
> > IW 10 [Indexer]: startCommit(): start sizeInBytes=0
> > IW 10 [Indexer]:   skip startCommit(): no changes pending
> > IW 10 [Indexer]: commit: pendingCommit == null; skip
> > IW 10 [Indexer]: commit: done
> > IW 10 [Indexer]: at close: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m
> >
> >
> > Peter
> >
> > On Thu, Feb 25, 2010 at 1:13 PM, Michael McCandless <
> > lucene@mikemccandless.com> wrote:
> >
> >> Do you know the place in the infoStream output where you got a reader
> >> with the wrong (unexplained extra +1) version?  If so, can you post
> >> the infoStream output up to that point?
> >>
> >> Mike
> >>
> >> On Thu, Feb 25, 2010 at 10:22 AM, Peter Keegan <peterlkeegan@gmail.com>
> >> wrote:
> >> > I've reproduced this and I have a bunch of infoStream log files. Since
> >> the
> >> > messages have no timestamps, it's hard to tell where the relevant
> entries
> >> > are. What should I be looking for?
> >> >
> >> > Peter
> >> >
> >> > On Mon, Feb 22, 2010 at 3:58 PM, Peter Keegan <peterlkeegan@gmail.com
> >> >wrote:
> >> >
> >> >> I'm pretty sure there are flushes and segment merges going on, but
as
> >> you
> >> >> said, that shouldn't affect the version increment. I'll see what I
> can
> >> do to
> >> >> get infoStream output.
> >> >>
> >> >> Thanks,
> >> >> Peter
> >> >>
> >> >>
> >> >> On Mon, Feb 22, 2010 at 2:30 PM, Michael McCandless <
> >> >> lucene@mikemccandless.com> wrote:
> >> >>
> >> >>> Well I'm at a loss then.  The version should only increment on
> commit.
> >> >>>
> >> >>> Can you make it all happen when infoStream is on, and post back?
> >> >>>
> >> >>> Mike
> >> >>>
> >> >>> On Mon, Feb 22, 2010 at 12:35 PM, Peter Keegan <
> peterlkeegan@gmail.com
> >> >
> >> >>> wrote:
> >> >>> > Only one writer thread and one writer process.
> >> >>> > I'm calling IndexWriter(Directory d, Analyzer a, boolean create,
> >> >>> > MaxFieldLength mfl), which sets autocommit=false.
> >> >>> >
> >> >>> > Peter
> >> >>> >
> >> >>> > On Mon, Feb 22, 2010 at 12:24 PM, Michael McCandless <
> >> >>> > lucene@mikemccandless.com> wrote:
> >> >>> >
> >> >>> >> That's curious.
> >> >>> >>
> >> >>> >> It's only on prepareCommit (or, commit, if you didn't
first
> prepare,
> >> >>> >> since that will call prepareCommit internally) that this
version
> >> >>> >> should increase.
> >> >>> >>
> >> >>> >> Is there only 1 thread doing this?
> >> >>> >>
> >> >>> >> Oh, and, are you passing false for autoCommit?
> >> >>> >>
> >> >>> >> Mike
> >> >>> >>
> >> >>> >> On Mon, Feb 22, 2010 at 11:43 AM, Peter Keegan <
> >> peterlkeegan@gmail.com
> >> >>> >
> >> >>> >> wrote:
> >> >>> >> > Using Lucene 2.9.1, I have the following pseudocode
which gets
> >> >>> repeated
> >> >>> >> at
> >> >>> >> > regular intervals:
> >> >>> >> >
> >> >>> >> > 1. FSDirectory dir = FSDirectory.open(java.io.File);
> >> >>> >> > 2. dir.setLockFactory(new SingleInstanceLockFactory());
> >> >>> >> > 3. IndexWriter writer = new IndexWriter(dir, Analyzer,
false,
> >> >>> >> maxFieldLen)
> >> >>> >> > 4. writer.getReader().getVersion();
> >> >>> >> > 5. writer.prepareCommit();
> >> >>> >> > 6. writer.getReader().getVersion();
> >> >>> >> > 7. writer.commit();
> >> >>> >> > 8. writer.close();
> >> >>> >> >
> >> >>> >> > I'm using the version number to keep external data
in synch
> with
> >> the
> >> >>> >> index.
> >> >>> >> > Usually, the version number from (6) is 1 greater
than from (4)
> >> and
> >> >>> the
> >> >>> >> > version from (4) equals the version from the previous
(6). At
> >> least
> >> >>> once
> >> >>> >> a
> >> >>> >> > day, however, the version from (4) is 1 greater than
from the
> >> >>> previous
> >> >>> >> (6).
> >> >>> >> > What would explain this sporadic behavior of version
numbers?
> >> >>> >> >
> >> >>> >> > Thanks,
> >> >>> >> > Peter
> >> >>> >> >
> >> >>> >>
> >> >>> >>
> >> ---------------------------------------------------------------------
> >> >>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >>> >> For additional commands, e-mail:
> java-user-help@lucene.apache.org
> >> >>> >>
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>>
> ---------------------------------------------------------------------
> >> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >> >>>
> >> >>>
> >> >>
> >> >
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message