From lucene-user-return-11624-apmail-jakarta-lucene-user-archive=jakarta.apache.org@jakarta.apache.org Fri Dec 03 21:02:08 2004 Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@www.apache.org Received: (qmail 83633 invoked from network); 3 Dec 2004 21:02:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (209.237.227.199) by minotaur-2.apache.org with SMTP; 3 Dec 2004 21:02:08 -0000 Received: (qmail 68232 invoked by uid 500); 3 Dec 2004 21:02:02 -0000 Delivered-To: apmail-jakarta-lucene-user-archive@jakarta.apache.org Received: (qmail 68213 invoked by uid 500); 3 Dec 2004 21:02:02 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 68198 invoked by uid 99); 3 Dec 2004 21:02:02 -0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FORGED_RCVD_HELO X-Spam-Check-By: apache.org Received-SPF: pass (hermes.apache.org: local policy) Received: from smtp-vbr2.xs4all.nl (HELO smtp-vbr2.xs4all.nl) (194.109.24.22) by apache.org (qpsmtpd/0.28) with ESMTP; Fri, 03 Dec 2004 13:02:00 -0800 Received: from k8l.lan (porta.xs4all.nl [80.127.24.69]) by smtp-vbr2.xs4all.nl (8.12.11/8.12.11) with ESMTP id iB3L1vY0042052 for ; Fri, 3 Dec 2004 22:01:58 +0100 (CET) (envelope-from paul.elschot@xs4all.nl) From: Paul Elschot To: lucene-user@jakarta.apache.org Subject: Re: IndexWriter.optimize and memory usage Date: Fri, 3 Dec 2004 22:01:57 +0100 User-Agent: KMail/1.5.4 References: <20041203050756.62308.qmail@web12706.mail.yahoo.com> <200412030843.07163.paul.elschot@xs4all.nl> In-Reply-To: <200412030843.07163.paul.elschot@xs4all.nl> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200412032201.57739.paul.elschot@xs4all.nl> X-Virus-Scanned: by XS4ALL Virus Scanner X-Virus-Checked: Checked X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N On Friday 03 December 2004 08:43, Paul Elschot wrote: > On Friday 03 December 2004 07:50, Chris Hostetter wrote: =2E.. > > So, If I'm understanding you (and the javadocs) correctly, the real key > > here is maxMergeDocs. =A0It seems like addDocument will never merge a > > segment untill maxMergeDocs have been added? ... meaning that I need a > > value less then the default (Integer.MAX_VALUE) if I want IndexWriter to > > do incrimental merges as I go ... > >=20 > > =A0=A0=A0=A0...except... > >=20 > > ...if that were the case, then exactly is the meaning of mergeFactor? > maxMergeDocs controls the sizes of the intermediate segments > when adding documents. > With maxMergeDocs at default, adding a document can take as much time as : (and have the same effect as) optimize. =A0Eg. with mergeFactor at 10, the > 1000'th added document will create a segment of size 1000. > With maxMergeDocs at a lower value than 1000, the last merge (of the 10 > segments with 100 docs each) will not be done. : optimize() uses mergeFactor for its final merges, but it ignores > maxMergeDocs.=20 Meanwhile these fields have been deprecated in the development version for set... methods. Setting minMergeDocs is is deprecated and to be replaced by setMaxBufferedDocs(). The javadoc for this reads: "Determines the minimal number of documents required before the buffered=20 in-memory documents are merging and a new Segment is created. Since Documen= ts=20 are merged in a RAMDirectory, large value gives faster indexing. At the sam= e=20 time, mergeFactor limits the number of files open in a FSDirectory." Regards, Paul Elschot --------------------------------------------------------------------- To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org For additional commands, e-mail: lucene-user-help@jakarta.apache.org