lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jibo John <jiboj...@mac.com>
Subject Re: ThreadedIndexWriter vs. IndexWriter
Date Fri, 31 Jul 2009 18:38:20 GMT
Number of docs are the same in the index for both the cases (200,000).
I haven't altered the benchmark/ code, but, used a profiler to verify  
that  Benchmark main thread is closed only after all other  threads  
are closed.

Thanks,
-Jibo


On Jul 31, 2009, at 2:34 AM, Michael McCandless wrote:

> Hmm... this doesn't sound right.
>
> That example (ThreadedIndexWriter) is meant to be a drop-in
> replacement, wherever you use an IndexWriter, that keeps an
> under-the-hood thread pool (using java.util.concurrent.*) to
> add/update documents with multiple threads.
>
> It should not result in a smaller index.
>
> Can you sanity check the index?  Eg is numDocs() the same for both?
> You definitely called close() on the writer, right?  That method waits
> for all threads to finish their work before actually closing.
>
> Mike
>
> On Thu, Jul 30, 2009 at 8:01 PM, Jibo John<jibojohn@mac.com> wrote:
>> While trying out a few tuning options using contrib/benchmak as  
>> described in
>> LIA (2nd edition) book, I had an interesting observation.
>>
>> If I use a ThreadedIndexWriter (picked the example from lia2e, page  
>> 356)
>> instead of IndexWriter, the index size got reduced by 40% compared  
>> to using
>> IndexWriter.
>> Index related configuration were the same for both the tests in the  
>> alg
>> file.
>>
>> I am curious how come using a threaded index writer will have an  
>> impact on
>> the index size.
>>
>> Appreciate your input.
>>
>> Thanks,
>> -Jibo
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message