lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@lucene.com>
Subject Re: Stress/scalability testing Lucene
Date Wed, 20 Nov 2002 22:48:02 GMT
Otis Gospodnetic wrote:
> Reding and writing at the same time is okay.  Only one thread can
> modify the index at a time.

Almost.  Only one process can modify it at a time, other processes will 
be prevented by the write.lock file.  Multiple threads can modify an 
index simultaneously.  The bulk of the work of the updates will be 
serialized by synchronization in IndexWriter, but the analysis of the 
text into tokens is parallelizeable.

Doug

> --- roy-lucene-user@xemaps.com wrote:
> 
>>Ah, for some reason i thought none of the Lucene methods were thread
>>safe,
>>or is this only in the case of reading and writing at the same time? 
>>I
>>thought I read this in the FAQ.
>>
>>Roy.
>>
>>-----Original Message-----
>>From: Doug Cutting [mailto:cutting@lucene.com]
>>Sent: Wednesday, November 20, 2002 5:04 PM
>>To: Lucene Users List
>>Subject: Re: Stress/scalability testing Lucene
>>
>>
>>* Replies will be sent through Spamex to
>>lucene-user@jakarta.apache.org
>>* For additional info click -> http://www.spamex.com/i/?v=886513
>>
>>Justin Greene wrote:
>>
>>>We created a thread pool to read and parse the email
>>>messages.  10 threads seems to be the magic number here for us.  We
>>
>>then
>>
>>>created a queue of messages to be indexed onto which we push the
>>
>>parsed
>>
>>>messages and have a single thread adding messages to the index.
>>
>>IndexWriter.addDocument(Document) is thread safe, so you don't need a
>>
>>separate indexing thread.  So long as your analyzer is thread safe,
>>you 
>>can index each messages in the thread that parses it, for even
>>greater 
>>parallelism.
>>
>>Doug
>>
>>
>>--
>>To unsubscribe, e-mail:
>><mailto:lucene-user-unsubscribe@jakarta.apache.org>
>>For additional commands, e-mail:
>><mailto:lucene-user-help@jakarta.apache.org>
>>
>>
>>This email and any attachments are confidential and may be 
>>legally privileged. No confidentiality or privilege is waived 
>>or lost by any transmission in error.  If you are not the 
>>intended recipient you are hereby notified that any use, 
>>printing, copying or disclosure is strictly prohibited.  
>>Please delete this email and any attachments, without 
>>printing, copying, forwarding or saving them and notify the 
>>sender immediately by reply e-mail.  Zurich Capital Markets 
>>and its affiliates reserve the right to monitor all e-mail 
>>communications through its networks.  Unless otherwise 
>>stated, any pricing information in this e-mail is indicative 
>>only, is subject to change and does not constitute an offer 
>>to enter into any transaction at such price and any terms in 
>>relation to any proposed transaction are indicative only and 
>>subject to express final confirmation.
>>
> 
> 
> 
> __________________________________________________
> Do you Yahoo!?
> Yahoo! Web Hosting - Let the expert host your site
> http://webhosting.yahoo.com
> 
> --
> To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>
> 



--
To unsubscribe, e-mail:   <mailto:lucene-user-unsubscribe@jakarta.apache.org>
For additional commands, e-mail: <mailto:lucene-user-help@jakarta.apache.org>


Mime
View raw message