Return-Path: Delivered-To: apmail-lucene-java-user-archive@www.apache.org Received: (qmail 37658 invoked from network); 26 Feb 2010 19:41:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 26 Feb 2010 19:41:31 -0000 Received: (qmail 60677 invoked by uid 500); 26 Feb 2010 19:41:29 -0000 Delivered-To: apmail-lucene-java-user-archive@lucene.apache.org Received: (qmail 60628 invoked by uid 500); 26 Feb 2010 19:41:29 -0000 Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: java-user@lucene.apache.org Delivered-To: mailing list java-user@lucene.apache.org Received: (qmail 60620 invoked by uid 99); 26 Feb 2010 19:41:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Feb 2010 19:41:29 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of peterlkeegan@gmail.com designates 209.85.222.196 as permitted sender) Received: from [209.85.222.196] (HELO mail-pz0-f196.google.com) (209.85.222.196) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 26 Feb 2010 19:41:22 +0000 Received: by pzk34 with SMTP id 34so350937pzk.3 for ; Fri, 26 Feb 2010 11:41:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=JLI1HGHS3BqXoYBukMafrEaxxXjTmXhcV0L+Hj9d1Vc=; b=FnoCPJNoMxGF+5iQRX4rpTQ5VPFMQX4/ynuVR3M2QU8SLJYuRrHrfTqiTcrx86V4Ms yhsciiyrV+tGnZQzgoeye0g5cOoGer/l/iqiDW39i60TO49wjwbE+PJHvX3y0yjS5ptS ryixw3LRIp3wWgoQAOJ9ipgYhmeUTEthtRr3c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=TJ+sT1Pk/gqmwGd2nvYIhBUBEJjMCVPSfpg6aHy8kzEq10NvpEzXkRwA8wQ42kQDTl sUOk3KSNCrmUXVeMH5wUsDZeeXNQ0H2/0vI///om3AlP0Rs3bBez0etz8NP7dZBUDGIu XDpr+PFJ103pmEHHaLd+RHd371X4sHmhhek/s= MIME-Version: 1.0 Received: by 10.142.74.13 with SMTP id w13mr487898wfa.28.1267213257032; Fri, 26 Feb 2010 11:40:57 -0800 (PST) In-Reply-To: <9ac0c6aa1002260940x2f326ad4ga7c254bd92ca3c7d@mail.gmail.com> References: <9ac0c6aa1002220924q75da08a4s741376ff876ed20c@mail.gmail.com> <9ac0c6aa1002221130w3aa9c59dq2fe627e85c22bcf3@mail.gmail.com> <9ac0c6aa1002251013j55a5e6e9h4a7e5331d2951c1b@mail.gmail.com> <9ac0c6aa1002260940x2f326ad4ga7c254bd92ca3c7d@mail.gmail.com> Date: Fri, 26 Feb 2010 14:40:56 -0500 Message-ID: Subject: Re: IndexWriter.getReader.getVersion behavior From: Peter Keegan To: java-user@lucene.apache.org Content-Type: multipart/alternative; boundary=001636e90f340a45eb048086147a --001636e90f340a45eb048086147a Content-Type: text/plain; charset=ISO-8859-1 Is there a way for the application to wait for the BG commit to finish before it calls IW.close? If so, would this prevent the extra version? The extra version causes the app. to think that the external data it committed is out of synch with the index, which requires the app to do extra processing to re-synch. Thanks, Peter On Fri, Feb 26, 2010 at 12:40 PM, Michael McCandless < lucene@mikemccandless.com> wrote: > OK -- I can now see what happened. > > There was a merge still running, when you called IW.commit (Lucene > Merge Thread #0). Because IW.commit does not wait for BG merges to > finish, but IW.close does (by default), this means you'll pick up an > extra version whenever a merge is running when you call close. > > Mike > > On Thu, Feb 25, 2010 at 2:52 PM, Peter Keegan > wrote: > > I'm pretty sure this output occurred when the version number skipped +1. > > The line containing '============'. separates the close/open IndexWriter. > > > > IFD [Indexer]: setInfoStream > > > deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@646f9dd9 > > IW 9 [Indexer]: setInfoStream: > > dir=org.apache.lucene.store.SimpleFSDirectory@ > pathname>\lresumes1.search.main.1 autoCommit=false > > > mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@5be44512mergeScheduler > =org.apache.lucene.index.ConcurrentMergeScheduler@6772cfdframBufferSizeMB > =16.0 > > maxBufferedDocs=-1 maxBuffereDeleteTerms=-1 > > maxFieldLength=100000 index=_a:C9780 _b:C1204->_b _c:C717->_b > _d:C1220->_d > > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h > > IW 9 [Indexer]: flush at getReader > > IW 9 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 9 [Indexer]: index before flush _a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > IW 9 [UpdWriterBuild : 9]: DW: RAM: now flush @ usedMB=15.816 > > allocMB=15.816 deletesMB=0.203 triggerMB=16 > > IW 9 [UpdWriterBuild : 9]: flush: segment=_j docStoreSegment=_j > > docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false > > numDocs=1456 numBufDelTerms=1456 > > IW 9 [UpdWriterBuild : 9]: index before flush _a:C9780 _b:C1204->_b > > _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f > _h:C1291->_h > > _i:C703->_h > > IW 9 [UpdWriterBuild : 9]: DW: flush postings as segment _j numDocs=1456 > > IW 9 [UpdWriterBuild : 9]: DW: oldRAMSize=16584704 > newFlushedSize=4969789 > > docs/MB=307.202 new/old=29.966% > > IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ; > > isCommit = false] > > IFD [UpdWriterBuild : 9]: now checkpoint "segments_b" [10 segments ; > > isCommit = false] > > IW 9 [UpdWriterBuild : 9]: LMP: findMerges: 10 segments > > IW 9 [UpdWriterBuild : 9]: LMP: level 6.863048 to 7.613048: 1 segments > > IW 9 [UpdWriterBuild : 9]: LMP: level 6.2247195 to 6.696363: 9 segments > > IW 9 [UpdWriterBuild : 9]: CMS: now merge > > IW 9 [UpdWriterBuild : 9]: CMS: index: _a:C9780 _b:C1204->_b > _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j > > IW 9 [UpdWriterBuild : 9]: CMS: no more merges pending; now return > > IW 9 [Indexer]: prepareCommit: flush > > IW 9 [Indexer]: flush: segment=_k docStoreSegment=_j > docStoreOffset=1456 > > flushDocs=true flushDeletes=true flushDocStores=true numDocs=509 > > numBufDelTerms=509 > > IW 9 [Indexer]: index before flush _a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j > > IW 9 [Indexer]: flush shared docStore segment _j > > IW 9 [Indexer]: DW: closeDocStore: 2 files to flush to segment _j > > numDocs=1965 > > IW 9 [Indexer]: DW: flush postings as segment _k numDocs=509 > > IW 9 [Indexer]: DW: oldRAMSize=7483392 newFlushedSize=1854970 > > docs/MB=287.727 new/old=24.788% > > IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit = > false] > > IW 9 [Indexer]: DW: apply 1965 buffered deleted terms and 0 deleted > docIDs > > and 0 deleted queries on 11 segments. > > IFD [Indexer]: now checkpoint "segments_b" [11 segments ; isCommit = > false] > > IW 9 [Indexer]: LMP: findMerges: 11 segments > > IW 9 [Indexer]: LMP: level 6.863048 to 7.613048: 1 segments > > IW 9 [Indexer]: LMP: level 6.2247195 to 6.696363: 10 segments > > IW 9 [Indexer]: LMP: 1 to 11: add this merge > > IW 9 [Indexer]: add merge to pendingMerges: _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j [total 1 pending] > > IW 9 [Indexer]: CMS: now merge > > IW 9 [Indexer]: CMS: index: _a:C9780 _b:C1204->_b _c:C717->_b > _d:C1220->_d > > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h > _j:C1456->_j > > _k:C509->_j > > IW 9 [Indexer]: CMS: consider merge _b:C1204->_b _c:C717->_b > _d:C1220->_d > > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h > _j:C1456->_j > > _k:C509->_j into _l [mergeDocStores] > > IW 9 [Indexer]: CMS: launch new thread [Lucene Merge Thread #0] > > IW 9 [Indexer]: CMS: no more merges pending; now return > > IW 9 [Indexer]: startCommit(): start sizeInBytes=0 > > IW 9 [Indexer]: startCommit index=_a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j changeCount=7 > > IW 9 [Lucene Merge Thread #0]: CMS: merge thread: start > > IW 9 [Indexer]: now sync _k.fnm > > IW 9 [Lucene Merge Thread #0]: now merge > > merge=_b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f > > _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j into _l > > [mergeDocStores] > > merge=org.apache.lucene.index.MergePolicy$OneMerge@5203ef34 > > index=_a:C9780 _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d > > _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h _j:C1456->_j > _k:C509->_j > > IW 9 [Lucene Merge Thread #0]: merging _b:C1204->_b _c:C717->_b > _d:C1220->_d > > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h > _j:C1456->_j > > _k:C509->_j into _l [mergeDocStores] > > IW 9 [Indexer]: now sync _k.tii > > IW 9 [Indexer]: now sync _k.tis > > IW 9 [Lucene Merge Thread #0]: merge: total 9909 docs > > IW 9 [Indexer]: now sync _j.tis > > IW 9 [Indexer]: now sync _j.tii > > IW 9 [Indexer]: now sync _j.fdx > > IW 9 [Indexer]: now sync _j.fdt > > IW 9 [Indexer]: now sync _k.prx > > IW 9 [Indexer]: now sync _k.nrm > > IW 9 [Indexer]: now sync _j.nrm > > IW 9 [Indexer]: now sync _j.prx > > IW 9 [Indexer]: now sync _j.fnm > > IW 9 [Indexer]: now sync _k.frq > > IW 9 [Indexer]: now sync _j.frq > > IW 9 [Indexer]: done all syncs > > IW 9 [Indexer]: flush at getReader > > IW 9 [Indexer]: flush: segment=null docStoreSegment=_k docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=true numDocs=0 > > numBufDelTerms=0 > > IW 9 [Indexer]: index before flush _a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j > > IW 9 [Indexer]: flush shared docStore segment _k > > IW 9 [Indexer]: DW: closeDocStore: 0 files to flush to segment _k > numDocs=0 > > IW 9 [Indexer]: commit: start > > IW 9 [Indexer]: commit: already prepared > > IW 9 [Indexer]: commit: pendingCommit != null > > IW 9 [Indexer]: commit: wrote segments file "segments_c" > > IFD [Indexer]: now checkpoint "segments_c" [11 segments ; isCommit = > true] > > IFD [Indexer]: deleteCommits: now decRef commit "segments_b" > > IFD [Indexer]: delete "segments_b" > > IW 9 [Indexer]: commit: done > > IW 9 [Indexer]: flush at getReader > > IW 9 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 9 [Indexer]: index before flush _a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j > > IW 9 [Indexer]: now flush at close > > IW 9 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 9 [Indexer]: index before flush _a:C9780 _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j > > IW 9 [Indexer]: CMS: now merge > > IW 9 [Indexer]: CMS: index: _a:C9780 _b:C1204->_b _c:C717->_b > _d:C1220->_d > > _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h _i:C703->_h > _j:C1456->_j > > _k:C509->_j > > IW 9 [Indexer]: CMS: no more merges pending; now return > > IW 9 [Lucene Merge Thread #0]: commitMerge: _b:C1204->_b _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j into _l [mergeDocStores] index=_a:C9780 > > _b:C1204->_b _c:C717->_b _d:C1220->_d _e:C778->_d _f:C1173->_f > _g:C858->_f > > _h:C1291->_h _i:C703->_h _j:C1456->_j _k:C509->_j > > IW 9 [Lucene Merge Thread #0]: commitMergeDeletes _b:C1204->_b > _c:C717->_b > > _d:C1220->_d _e:C778->_d _f:C1173->_f _g:C858->_f _h:C1291->_h > _i:C703->_h > > _j:C1456->_j _k:C509->_j into _l [mergeDocStores] > > IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ; > > isCommit = false] > > IFD [Lucene Merge Thread #0]: now checkpoint "segments_c" [2 segments ; > > isCommit = false] > > IW 9 [Lucene Merge Thread #0]: CMS: merge thread: done > > IW 9 [Indexer]: now call final commit() > > IW 9 [Indexer]: startCommit(): start sizeInBytes=0 > > IW 9 [Indexer]: startCommit index=_a:C9780 _l:C9909 changeCount=8 > > IW 9 [Indexer]: now sync _l.nrm > > IW 9 [Indexer]: now sync _l.fnm > > IW 9 [Indexer]: now sync _l.prx > > IW 9 [Indexer]: now sync _l.frq > > IW 9 [Indexer]: now sync _l.tis > > IW 9 [Indexer]: now sync _l.fdx > > IW 9 [Indexer]: now sync _l.fdt > > IW 9 [Indexer]: now sync _l.tii > > IW 9 [Indexer]: done all syncs > > IW 9 [Indexer]: commit: pendingCommit != null > > IW 9 [Indexer]: commit: wrote segments file "segments_d" > > IFD [Indexer]: now checkpoint "segments_d" [2 segments ; isCommit = true] > > IFD [Indexer]: deleteCommits: now decRef commit "segments_c" > > IFD [Indexer]: delete "_h.prx" > > IFD [Indexer]: delete "_f.tis" > > IFD [Indexer]: delete "_g.tis" > > IFD [Indexer]: delete "_d.tii" > > IFD [Indexer]: delete "_e.frq" > > IFD [Indexer]: delete "_k.fnm" > > IFD [Indexer]: delete "_c.tii" > > IFD [Indexer]: delete "_h.nrm" > > IFD [Indexer]: delete "_i.tis" > > IFD [Indexer]: delete "_k.tii" > > IFD [Indexer]: delete "_h.fdt" > > IFD [Indexer]: delete "_c.tis" > > IFD [Indexer]: delete "_g.tii" > > IFD [Indexer]: delete "_h.fdx" > > IFD [Indexer]: delete "_c.nrm" > > IFD [Indexer]: delete "_i.tii" > > IFD [Indexer]: delete "_d.tis" > > IFD [Indexer]: delete "_f.tii" > > IFD [Indexer]: delete "_e.fnm" > > IFD [Indexer]: delete "_k.tis" > > IFD [Indexer]: delete "_d.frq" > > IFD [Indexer]: delete "_f.nrm" > > IFD [Indexer]: delete "_h.fnm" > > IFD [Indexer]: delete "segments_c" > > IFD [Indexer]: delete "_c.prx" > > IFD [Indexer]: delete "_d.fnm" > > IFD [Indexer]: delete "_h.frq" > > IFD [Indexer]: delete "_h.tis" > > IFD [Indexer]: delete "_f.fdt" > > IFD [Indexer]: delete "_j.tis" > > IFD [Indexer]: delete "_f.fdx" > > IFD [Indexer]: delete "_g.nrm" > > IFD [Indexer]: delete "_f.prx" > > IFD [Indexer]: delete "_j.tii" > > IFD [Indexer]: delete "_h.tii" > > IFD [Indexer]: delete "_e.prx" > > IFD [Indexer]: delete "_c.frq" > > IFD [Indexer]: delete "_i.prx" > > IFD [Indexer]: delete "_j.fdx" > > IFD [Indexer]: delete "_i.nrm" > > IFD [Indexer]: delete "_j.fdt" > > IFD [Indexer]: delete "_k.prx" > > IFD [Indexer]: delete "_b.nrm" > > IFD [Indexer]: delete "_e.tis" > > IFD [Indexer]: delete "_e.tii" > > IFD [Indexer]: delete "_g.fnm" > > IFD [Indexer]: delete "_k.nrm" > > IFD [Indexer]: delete "_j.nrm" > > IFD [Indexer]: delete "_j.prx" > > IFD [Indexer]: delete "_e.nrm" > > IFD [Indexer]: delete "_g.prx" > > IFD [Indexer]: delete "_b.prx" > > IFD [Indexer]: delete "_i.frq" > > IFD [Indexer]: delete "_b.tis" > > IFD [Indexer]: delete "_d.prx" > > IFD [Indexer]: delete "_g.frq" > > IFD [Indexer]: delete "_d.fdx" > > IFD [Indexer]: delete "_b.tii" > > IFD [Indexer]: delete "_b.frq" > > IFD [Indexer]: delete "_d.fdt" > > IFD [Indexer]: delete "_f.fnm" > > IFD [Indexer]: delete "_j.fnm" > > IFD [Indexer]: delete "_b.fnm" > > IFD [Indexer]: delete "_d.nrm" > > IFD [Indexer]: delete "_b.fdt" > > IFD [Indexer]: delete "_k.frq" > > IFD [Indexer]: delete "_c.fnm" > > IFD [Indexer]: delete "_j.frq" > > IFD [Indexer]: delete "_i.fnm" > > IFD [Indexer]: delete "_f.frq" > > IFD [Indexer]: delete "_b.fdx" > > IW 9 [Indexer]: commit: done > > IW 9 [Indexer]: at close: _a:C9780 _l:C9909 > > > > ===================================================== > > > > IFD [Indexer]: setInfoStream > > > deletionPolicy=org.apache.lucene.index.KeepOnlyLastCommitDeletionPolicy@68d349c1 > > IW 10 [Indexer]: setInfoStream: > > dir=org.apache.lucene.store.SimpleFSDirectory@ > pathname>\lresumes1.search.main.1 autoCommit=false > > > mergePolicy=org.apache.lucene.index.LogByteSizeMergePolicy@78f53abmergeScheduler > =org.apache.lucene.index.ConcurrentMergeScheduler@52be036dramBufferSizeMB > =16.0 > > maxBufferedDocs=-1 maxBuffereDeleteTerms=-1 > > maxFieldLength=100000 index=_a:C9780 _l:C9909 > > IW 10 [Indexer]: flush at getReader > > IW 10 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 10 [Indexer]: index before flush _a:C9780 _l:C9909 > > IW 10 [UpdWriterBuild : 10]: DW: RAM: now flush @ usedMB=15.789 > > allocMB=15.789 deletesMB=0.212 triggerMB=16 > > IW 10 [UpdWriterBuild : 10]: flush: segment=_m docStoreSegment=_m > > docStoreOffset=0 flushDocs=true flushDeletes=false flushDocStores=false > > numDocs=1526 numBufDelTerms=1526 > > IW 10 [UpdWriterBuild : 10]: index before flush _a:C9780 _l:C9909 > > IW 10 [UpdWriterBuild : 10]: DW: flush postings as segment _m > numDocs=1526 > > IW 10 [UpdWriterBuild : 10]: DW: oldRAMSize=16556032 > > newFlushedSize=4788823 docs/MB=334.138 new/old=28.925% > > IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ; > > isCommit = false] > > IFD [UpdWriterBuild : 10]: now checkpoint "segments_d" [3 segments ; > > isCommit = false] > > IW 10 [UpdWriterBuild : 10]: LMP: findMerges: 3 segments > > IW 10 [UpdWriterBuild : 10]: LMP: level 6.8642592 to 7.6142592: 2 > segments > > IW 10 [UpdWriterBuild : 10]: LMP: level 6.2247195 to 6.6802545: 1 > segments > > IW 10 [UpdWriterBuild : 10]: CMS: now merge > > IW 10 [UpdWriterBuild : 10]: CMS: index: _a:C9780 _l:C9909 _m:C1526->_m > > IW 10 [UpdWriterBuild : 10]: CMS: no more merges pending; now return > > IW 10 [Indexer]: prepareCommit: flush > > IW 10 [Indexer]: flush: segment=_n docStoreSegment=_m > docStoreOffset=1526 > > flushDocs=true flushDeletes=true flushDocStores=true numDocs=501 > > numBufDelTerms=501 > > IW 10 [Indexer]: index before flush _a:C9780 _l:C9909 _m:C1526->_m > > IW 10 [Indexer]: flush shared docStore segment _m > > IW 10 [Indexer]: DW: closeDocStore: 2 files to flush to segment _m > > numDocs=2027 > > IW 10 [Indexer]: DW: flush postings as segment _n numDocs=501 > > IW 10 [Indexer]: DW: oldRAMSize=6927360 newFlushedSize=1659342 > > docs/MB=316.593 new/old=23.953% > > IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit = > false] > > IW 10 [Indexer]: DW: apply 2027 buffered deleted terms and 0 deleted > docIDs > > and 0 deleted queries on 4 segments. > > IFD [Indexer]: now checkpoint "segments_d" [4 segments ; isCommit = > false] > > IW 10 [Indexer]: LMP: findMerges: 4 segments > > IW 10 [Indexer]: LMP: level 6.8642592 to 7.6142592: 2 segments > > IW 10 [Indexer]: LMP: level 6.2247195 to 6.6802545: 1 segments > > IW 10 [Indexer]: LMP: level -1.0 to 6.220011: 1 segments > > IW 10 [Indexer]: CMS: now merge > > IW 10 [Indexer]: CMS: index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m > > IW 10 [Indexer]: CMS: no more merges pending; now return > > IW 10 [Indexer]: startCommit(): start sizeInBytes=0 > > IW 10 [Indexer]: startCommit index=_a:C9780 _l:C9909 _m:C1526->_m > > _n:C501->_m changeCount=6 > > IW 10 [Indexer]: now sync _m.frq > > IW 10 [Indexer]: now sync _m.prx > > IW 10 [Indexer]: now sync _n.tis > > IW 10 [Indexer]: now sync _m.tis > > IW 10 [Indexer]: now sync _n.frq > > IW 10 [Indexer]: now sync _n.tii > > IW 10 [Indexer]: now sync _m.tii > > IW 10 [Indexer]: now sync _m.fnm > > IW 10 [Indexer]: now sync _n.fnm > > IW 10 [Indexer]: now sync _n.nrm > > IW 10 [Indexer]: now sync _n.prx > > IW 10 [Indexer]: now sync _m.nrm > > IW 10 [Indexer]: now sync _m.fdt > > IW 10 [Indexer]: now sync _m.fdx > > IW 10 [Indexer]: done all syncs > > IW 10 [Indexer]: flush at getReader > > IW 10 [Indexer]: flush: segment=null docStoreSegment=_n > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=true numDocs=0 > > numBufDelTerms=0 > > IW 10 [Indexer]: index before flush _a:C9780 _l:C9909 _m:C1526->_m > > _n:C501->_m > > IW 10 [Indexer]: flush shared docStore segment _n > > IW 10 [Indexer]: DW: closeDocStore: 0 files to flush to segment _n > numDocs=0 > > IW 10 [Indexer]: commit: start > > IW 10 [Indexer]: commit: already prepared > > IW 10 [Indexer]: commit: pendingCommit != null > > IW 10 [Indexer]: commit: wrote segments file "segments_e" > > IFD [Indexer]: now checkpoint "segments_e" [4 segments ; isCommit = true] > > IFD [Indexer]: deleteCommits: now decRef commit "segments_d" > > IFD [Indexer]: delete "segments_d" > > IW 10 [Indexer]: commit: done > > IW 10 [Indexer]: flush at getReader > > IW 10 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 10 [Indexer]: index before flush _a:C9780 _l:C9909 _m:C1526->_m > > _n:C501->_m > > IW 10 [Indexer]: now flush at close > > IW 10 [Indexer]: flush: segment=null docStoreSegment=null > docStoreOffset=0 > > flushDocs=false flushDeletes=true flushDocStores=false numDocs=0 > > numBufDelTerms=0 > > IW 10 [Indexer]: index before flush _a:C9780 _l:C9909 _m:C1526->_m > > _n:C501->_m > > IW 10 [Indexer]: CMS: now merge > > IW 10 [Indexer]: CMS: index: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m > > IW 10 [Indexer]: CMS: no more merges pending; now return > > IW 10 [Indexer]: now call final commit() > > IW 10 [Indexer]: startCommit(): start sizeInBytes=0 > > IW 10 [Indexer]: skip startCommit(): no changes pending > > IW 10 [Indexer]: commit: pendingCommit == null; skip > > IW 10 [Indexer]: commit: done > > IW 10 [Indexer]: at close: _a:C9780 _l:C9909 _m:C1526->_m _n:C501->_m > > > > > > Peter > > > > On Thu, Feb 25, 2010 at 1:13 PM, Michael McCandless < > > lucene@mikemccandless.com> wrote: > > > >> Do you know the place in the infoStream output where you got a reader > >> with the wrong (unexplained extra +1) version? If so, can you post > >> the infoStream output up to that point? > >> > >> Mike > >> > >> On Thu, Feb 25, 2010 at 10:22 AM, Peter Keegan > >> wrote: > >> > I've reproduced this and I have a bunch of infoStream log files. Since > >> the > >> > messages have no timestamps, it's hard to tell where the relevant > entries > >> > are. What should I be looking for? > >> > > >> > Peter > >> > > >> > On Mon, Feb 22, 2010 at 3:58 PM, Peter Keegan >> >wrote: > >> > > >> >> I'm pretty sure there are flushes and segment merges going on, but as > >> you > >> >> said, that shouldn't affect the version increment. I'll see what I > can > >> do to > >> >> get infoStream output. > >> >> > >> >> Thanks, > >> >> Peter > >> >> > >> >> > >> >> On Mon, Feb 22, 2010 at 2:30 PM, Michael McCandless < > >> >> lucene@mikemccandless.com> wrote: > >> >> > >> >>> Well I'm at a loss then. The version should only increment on > commit. > >> >>> > >> >>> Can you make it all happen when infoStream is on, and post back? > >> >>> > >> >>> Mike > >> >>> > >> >>> On Mon, Feb 22, 2010 at 12:35 PM, Peter Keegan < > peterlkeegan@gmail.com > >> > > >> >>> wrote: > >> >>> > Only one writer thread and one writer process. > >> >>> > I'm calling IndexWriter(Directory d, Analyzer a, boolean create, > >> >>> > MaxFieldLength mfl), which sets autocommit=false. > >> >>> > > >> >>> > Peter > >> >>> > > >> >>> > On Mon, Feb 22, 2010 at 12:24 PM, Michael McCandless < > >> >>> > lucene@mikemccandless.com> wrote: > >> >>> > > >> >>> >> That's curious. > >> >>> >> > >> >>> >> It's only on prepareCommit (or, commit, if you didn't first > prepare, > >> >>> >> since that will call prepareCommit internally) that this version > >> >>> >> should increase. > >> >>> >> > >> >>> >> Is there only 1 thread doing this? > >> >>> >> > >> >>> >> Oh, and, are you passing false for autoCommit? > >> >>> >> > >> >>> >> Mike > >> >>> >> > >> >>> >> On Mon, Feb 22, 2010 at 11:43 AM, Peter Keegan < > >> peterlkeegan@gmail.com > >> >>> > > >> >>> >> wrote: > >> >>> >> > Using Lucene 2.9.1, I have the following pseudocode which gets > >> >>> repeated > >> >>> >> at > >> >>> >> > regular intervals: > >> >>> >> > > >> >>> >> > 1. FSDirectory dir = FSDirectory.open(java.io.File); > >> >>> >> > 2. dir.setLockFactory(new SingleInstanceLockFactory()); > >> >>> >> > 3. IndexWriter writer = new IndexWriter(dir, Analyzer, false, > >> >>> >> maxFieldLen) > >> >>> >> > 4. writer.getReader().getVersion(); > >> >>> >> > 5. writer.prepareCommit(); > >> >>> >> > 6. writer.getReader().getVersion(); > >> >>> >> > 7. writer.commit(); > >> >>> >> > 8. writer.close(); > >> >>> >> > > >> >>> >> > I'm using the version number to keep external data in synch > with > >> the > >> >>> >> index. > >> >>> >> > Usually, the version number from (6) is 1 greater than from (4) > >> and > >> >>> the > >> >>> >> > version from (4) equals the version from the previous (6). At > >> least > >> >>> once > >> >>> >> a > >> >>> >> > day, however, the version from (4) is 1 greater than from the > >> >>> previous > >> >>> >> (6). > >> >>> >> > What would explain this sporadic behavior of version numbers? > >> >>> >> > > >> >>> >> > Thanks, > >> >>> >> > Peter > >> >>> >> > > >> >>> >> > >> >>> >> > >> --------------------------------------------------------------------- > >> >>> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > >> >>> >> For additional commands, e-mail: > java-user-help@lucene.apache.org > >> >>> >> > >> >>> >> > >> >>> > > >> >>> > >> >>> > --------------------------------------------------------------------- > >> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > >> >>> For additional commands, e-mail: java-user-help@lucene.apache.org > >> >>> > >> >>> > >> >> > >> > > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > >> For additional commands, e-mail: java-user-help@lucene.apache.org > >> > >> > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org > For additional commands, e-mail: java-user-help@lucene.apache.org > > --001636e90f340a45eb048086147a--