lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: shared instance of IndexWriter doesn't improve proformance
Date Wed, 11 Jan 2012 12:08:16 GMT
I think it's hard to compare the results here?

In test 1 (single IW shared across threads) you end up with one index.

In test 2 (private IW per thread) you end up with N indexes, which to
be "fair" need to be merged down into one index (eg with .addIndexes)?

Or seen another way, test 1 should have a much larger merge factor to
be comparable to test 2.

You should also call IW.close(false) (aborts all running merges)
whenever comparing two indexing runs... otherwise a long running merge
can completely skew the results.

However, there is known contention in IndexWriter's flushing (when it
writes the buffered documents to a new segment), which is still
present in 3.x but is fixed on trunk (to be 4.0)... see this post:

  http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html

Mike McCandless

http://blog.mikemccandless.com



On Tue, Jan 10, 2012 at 8:32 PM, Cheng <zhoucheng2008@gmail.com> wrote:
> Hi,
>
> I use a same instance of writer for multiple threads. It turns out that the
> time to finish jobs is more than to create a new writer instance in each
> thread. What would be the possible reasons?
>
> Thanks

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message