Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@apache.org Received: (qmail 48056 invoked from network); 20 Nov 2002 22:01:47 -0000 Received: from unknown (HELO nagoya.betaversion.org) (192.18.49.131) by daedalus.apache.org with SMTP; 20 Nov 2002 22:01:47 -0000 Received: (qmail 5819 invoked by uid 97); 20 Nov 2002 22:02:49 -0000 Delivered-To: qmlist-jakarta-archive-lucene-user@jakarta.apache.org Received: (qmail 5801 invoked by uid 97); 20 Nov 2002 22:02:48 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 5782 invoked by uid 98); 20 Nov 2002 22:02:47 -0000 X-Antivirus: nagoya (v4218 created Aug 14 2002) Message-ID: <3DDC06C0.6030204@lucene.com> Date: Wed, 20 Nov 2002 14:03:44 -0800 From: Doug Cutting User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Lucene Users List Subject: Re: Stress/scalability testing Lucene References: <013445F6BB17D4119959005004AAEA9A8B10A1@SPIDERMAN> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Justin Greene wrote: > We created a thread pool to read and parse the email > messages. 10 threads seems to be the magic number here for us. We then > created a queue of messages to be indexed onto which we push the parsed > messages and have a single thread adding messages to the index. IndexWriter.addDocument(Document) is thread safe, so you don't need a separate indexing thread. So long as your analyzer is thread safe, you can index each messages in the thread that parses it, for even greater parallelism. Doug -- To unsubscribe, e-mail: For additional commands, e-mail: