lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: IndexSearcher and multi-threaded performance
Date Wed, 12 Nov 2008 22:31:17 GMT

Good!

In fact now we see similar slowness with nio-thread vs nio-shared as  
we see for RAM-thread vs RAM-shared.  Ie, for both RAM and NIO you get  
better performance sharing a single reader than reader-per-thread.   
This is odd -- I would have expected that with infinite RAM reader-per- 
thread would always be as fast or faster than a shared reader.  It's  
particularly interesting that the RAM case shows it worse than the NIO  
case.

Maybe it's because in the reader-per-thread case the CPU's cache is  
less effective since things like norms & deleted docs are now  
replicated in memory?

Mike

Dmitri Bichko wrote:

> Nice!
>
> At 8 threads nio-shared catches up with ram-shared.  Here's the  
> complete table:
>
> 	fs-thread	nio-thread	ram-thread	fs-shared	nio-shared	ram-shared
> 1	71877	70461	54739	73986	72155	61595
> 2	34949	34945	26735	43719	33019	28935
> 3	25581	28732	26885	38412	23383	19624
> 4	20511	21235	31742	38712	18000	15059
> 5	19235	21060	24345	39685	14636	12509
> 6	16775	17685	26896	39592	12649	10841
> 7	17147	18766	18296	46678	11201	10183
> 8	18327	17588	19043	39886	10439	10048
> 9	16885	16483	18721	40342	9455	9483
> 10	17832	17428	30757	44706	8947	10975
> 11	17251	16405	21199	39947	8597	9704
> 12	17267	17967	36284	40208	8462	10996
>
> And it behaves very well with more threads:
>
> 	nio-shared
> 1	71066
> 2	33206
> 3	22824
> 4	18168
> 5	15198
> 6	13086
> 7	11616
> 8	10698
> 9	9919
> 10	9657
> 11	9409
> 12	8977
> 13	9210
> 14	8757
> 15	9282
> 16	9260
> 17	9010
> 18	8230
> 19	8439
> 20	8486
> 21	8631
> 22	8417
> 23	8154
> 24	8685
> 25	7878
> 26	8398
> 27	8265
> 28	8266
> 29	7951
> 30	8606
> 31	8385
> 32	8630
>
> That solves it for me, but I do see a fair amount of free time on this
> machine - if there are other things you want to benchmark, I'd be
> happy to do it.
>
> Cheers,
> Dmitri
>
> On Tue, Nov 11, 2008 at 9:43 PM, Mark Miller <markrmiller@gmail.com>  
> wrote:
>> Mark Miller wrote:
>>>
>>> Thats a good point, and points out a bug in solr trunk for me.  
>>> Frankly I
>>> don't see how its done. There is no code I can see/find to use it  
>>> rather
>>> than FSDirectory. Still assuming there must be a way, but I don't  
>>> see it...
>>>
>> Ah - brain freeze. What else is new :) You have to set the system  
>> property
>> to change implementations: org.apache.lucene.FSDirectory.class is the
>> property, set it to the class. Been a long time...
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message