Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@www.apache.org Received: (qmail 62693 invoked from network); 20 Apr 2004 08:41:03 -0000 Received: from daedalus.apache.org (HELO mail.apache.org) (208.185.179.12) by minotaur-2.apache.org with SMTP; 20 Apr 2004 08:41:03 -0000 Received: (qmail 87420 invoked by uid 500); 20 Apr 2004 08:40:32 -0000 Delivered-To: apmail-jakarta-lucene-user-archive@jakarta.apache.org Received: (qmail 87390 invoked by uid 500); 20 Apr 2004 08:40:31 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 87377 invoked from network); 20 Apr 2004 08:40:31 -0000 Received: from unknown (HELO dev.sofari.com) (12.43.53.196) by daedalus.apache.org with SMTP; 20 Apr 2004 08:40:31 -0000 Received: from newsmonster.org (dsl093-132-107.sfo4.dsl.speakeasy.net [66.93.132.107]) (authenticated bits=0) by dev.sofari.com (8.12.10/8.12.10) with ESMTP id i3K8eltG015737 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 20 Apr 2004 01:40:48 -0700 Message-ID: <4084E209.7060500@newsmonster.org> Date: Tue, 20 Apr 2004 01:40:41 -0700 From: "Kevin A. Burton" User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en MIME-Version: 1.0 To: Lucene Users List Subject: Does a RAMDirectory ever need to merge segments... (performance issue) X-Enigmail-Version: 0.83.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig1DB07C2BA6D56319C0B7A1E6" X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: minotaur-2.apache.org 1.6.2 0/1000/N --------------enig1DB07C2BA6D56319C0B7A1E6 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I've been benchmarking our indexer to find out if I can squeeze any more performance out of it. I noticed one problem with RAMDirectory... I'm storing documents in memory and then writing them to disk every once in a while. ... IndexWriter.maybeMergeSegments is taking up 5% of total runtime. DocumentWriter.addDocument is taking up another 17% of total runtime. Notice that this doesn't == 100% becuase there are other tasks taking up CPU before and after Lucene is called. Anyway... I don't see why RAMDirectory is trying to merge segments. Is there anyway to prevent this? I could just store them in a big ArrayList until I'm ready to write them to a disk index but I'm not sure how efficient this will be. Anyone run into this before? -- Please reply using PGP. http://peerfear.org/pubkey.asc NewsMonster - http://www.newsmonster.org/ Kevin A. Burton, Location - San Francisco, CA, Cell - 415.595.9965 AIM/YIM - sfburtonator, Web - http://peerfear.org/ GPG fingerprint: 5FB2 F3E2 760E 70A8 6174 D393 E84D 8D04 99F1 4412 IRC - freenode.net #infoanarchy | #p2p-hackers | #newsmonster --------------enig1DB07C2BA6D56319C0B7A1E6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFAhOIL6E2NBJnxRBIRApp5AKDN+6nHf3iYQuvQdD/ELKIP8CBuhACgsC/t VjGnuqkCKp1jvYEPIY6gf7E= =eMVb -----END PGP SIGNATURE----- --------------enig1DB07C2BA6D56319C0B7A1E6--