lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: Unable to improve performance
Date Fri, 27 Mar 2009 11:32:10 GMT
Are you opening your IndexReader with readOnly=true?  If not, you're
likely hitting contention on the "isDeleted" method.

When you run with a "normal" directory, either on a traditional hard
drive or SSD device, do you use NIOFSDirectory?  That removes
contention, but, it only works on non-Windows platform due to a
long-standing bug in Sun's JRE.

Likely the OS is caching stuff in RAM, anyway, so you don't see much
improvement when you explicitly load into a RAMDir.

Mike

On Fri, Mar 27, 2009 at 7:07 AM, Paul Taylor <paul_t100@fastmail.fm> wrote:
> Hi
>
> I am trying to run the performance tests against lucene, and am suprised
> about the results.
>
> I have a test that creates a queue of queries, and a number of threads. The
> threads run concurrently getting the next query available, peforming a query
> on the index and taking the top hits. The index is 2GB in size, and was
> originally created froma database table of about 7 millions rows.
>
> I ran the test a number of times with 30 threads, and max memory of 3500mb I
> was processing 10,000 records in about 43 seconds ( 233 queries/second) ,
> the index was stored on a solid state drive running on a MacBook Pro (2.66
> Ghz Intel Core 2 Duo, 4GB DDR). I dont really have a view on whether this is
> a good result or not but I was keen to try a few other things to see if I
> could improve performance further, but all my efforts have had minimal
> effect.
>
> I tried creating a RAMDirectory based on the file index, once the index had
> been created (4 min 20 seconds) it again took
> I copied the index to a slower external convention hard drive and it still
> took 43 seconds.
>
> Reducing/increasing the memory allocated and the number of threads had
> minimal impact.
>
> The main thing Im suprised about is I was expecting a massive difference in
> holding the index in memory instead on disk
>
> thanks Paul
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message