lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toke Eskildsen ...@statsbiblioteket.dk>
Subject Re: Best practices for searcher memory usage?
Date Wed, 14 Jul 2010 08:13:48 GMT
On Tue, 2010-07-13 at 23:49 +0200, Christopher Condit wrote:
> * 20 million documents [...]
> * 140GB total index size
> * Optimized into a single segment

I take it that you do not have frequent updates? Have you tried to see
if you can get by with more segments without significant slowdown?

> The application will run with 10G of -Xmx but any less and it bails out. 
> It seems happier if we feed it 12GB. The searches are starting to bog 
> down a bit (5-10 seconds for some queries)...

10G sounds like a lot for that index. Two common memory-eaters are
sorting by field value and faceting. Could you describe what you're
doing in that regard?

Similarly, the 5-10 seconds for some queries seems very slow. Could you
give some examples on the queries that causes problems together with
some examples of fast queries and how long they take to execute?


The standard silver bullet for easy performance boost is to buy a couple
of consumer grade SSDs and put them on the local machine. If you're
gearing up to use more machines you might want to try this first.

Regards,
Toke


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message