Return-Path: Delivered-To: apmail-jakarta-lucene-user-archive@apache.org Received: (qmail 89758 invoked from network); 20 Nov 2002 22:46:04 -0000 Received: from unknown (HELO nagoya.betaversion.org) (192.18.49.131) by daedalus.apache.org with SMTP; 20 Nov 2002 22:46:04 -0000 Received: (qmail 9156 invoked by uid 97); 20 Nov 2002 22:47:06 -0000 Delivered-To: qmlist-jakarta-archive-lucene-user@jakarta.apache.org Received: (qmail 9137 invoked by uid 97); 20 Nov 2002 22:47:06 -0000 Mailing-List: contact lucene-user-help@jakarta.apache.org; run by ezmlm Precedence: bulk List-Unsubscribe: List-Subscribe: List-Help: List-Post: List-Id: "Lucene Users List" Reply-To: "Lucene Users List" Delivered-To: mailing list lucene-user@jakarta.apache.org Received: (qmail 9125 invoked by uid 98); 20 Nov 2002 22:47:05 -0000 X-Antivirus: nagoya (v4218 created Aug 14 2002) Message-ID: <3DDC1122.4070901@lucene.com> Date: Wed, 20 Nov 2002 14:48:02 -0800 From: Doug Cutting User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.1) Gecko/20021003 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Lucene Users List Subject: Re: Stress/scalability testing Lucene References: <20021120221614.9960.qmail@web12703.mail.yahoo.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N X-Spam-Rating: daedalus.apache.org 1.6.2 0/1000/N Otis Gospodnetic wrote: > Reding and writing at the same time is okay. Only one thread can > modify the index at a time. Almost. Only one process can modify it at a time, other processes will be prevented by the write.lock file. Multiple threads can modify an index simultaneously. The bulk of the work of the updates will be serialized by synchronization in IndexWriter, but the analysis of the text into tokens is parallelizeable. Doug > --- roy-lucene-user@xemaps.com wrote: > >>Ah, for some reason i thought none of the Lucene methods were thread >>safe, >>or is this only in the case of reading and writing at the same time? >>I >>thought I read this in the FAQ. >> >>Roy. >> >>-----Original Message----- >>From: Doug Cutting [mailto:cutting@lucene.com] >>Sent: Wednesday, November 20, 2002 5:04 PM >>To: Lucene Users List >>Subject: Re: Stress/scalability testing Lucene >> >> >>* Replies will be sent through Spamex to >>lucene-user@jakarta.apache.org >>* For additional info click -> http://www.spamex.com/i/?v=886513 >> >>Justin Greene wrote: >> >>>We created a thread pool to read and parse the email >>>messages. 10 threads seems to be the magic number here for us. We >> >>then >> >>>created a queue of messages to be indexed onto which we push the >> >>parsed >> >>>messages and have a single thread adding messages to the index. >> >>IndexWriter.addDocument(Document) is thread safe, so you don't need a >> >>separate indexing thread. So long as your analyzer is thread safe, >>you >>can index each messages in the thread that parses it, for even >>greater >>parallelism. >> >>Doug >> >> >>-- >>To unsubscribe, e-mail: >> >>For additional commands, e-mail: >> >> >> >>This email and any attachments are confidential and may be >>legally privileged. No confidentiality or privilege is waived >>or lost by any transmission in error. If you are not the >>intended recipient you are hereby notified that any use, >>printing, copying or disclosure is strictly prohibited. >>Please delete this email and any attachments, without >>printing, copying, forwarding or saving them and notify the >>sender immediately by reply e-mail. Zurich Capital Markets >>and its affiliates reserve the right to monitor all e-mail >>communications through its networks. Unless otherwise >>stated, any pricing information in this e-mail is indicative >>only, is subject to change and does not constitute an offer >>to enter into any transaction at such price and any terms in >>relation to any proposed transaction are indicative only and >>subject to express final confirmation. >> > > > > __________________________________________________ > Do you Yahoo!? > Yahoo! Web Hosting - Let the expert host your site > http://webhosting.yahoo.com > > -- > To unsubscribe, e-mail: > For additional commands, e-mail: > -- To unsubscribe, e-mail: For additional commands, e-mail: