lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: Indexing in multi-threaded environment
Date Tue, 10 May 2005 19:50:29 GMT
Chris Lamprecht wrote:
> I've done exactly what you describe, using N threads where N is the
> number of processors on the machine, plus one more thread that writes
> to the file system index (since that is I/O-bound anyway).  Since most
> of the CPU time is tokenizing/stemming/etc, the method works well. 
> The main drawback is IndexWriter.addIndexes(Directory[]) always calls
> optimize, which takes a lot of time as the index grows.

IndexWriter.addDocument() is thread-safe.  So you can simply have 
multiple threads which all add documents to a single IndexWriter. 
Tokenization is not synchronized, so multiple threads will operate in 
parallel.

Doug

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message