lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Approximately needed RAM for 5000 query/second at a Solr machine?
Date Wed, 10 Apr 2013 00:50:45 GMT
On 4/9/2013 4:06 PM, Furkan KAMACI wrote:
> Are there anybody who can help me about how to guess the approximately
> needed RAM for 5000 query/second at a Solr machine?

You've already gotten some good replies, and I'm aware that they haven't 
really answered your question.  This is the kind of question that cannot 
be answered.

The amount of RAM that you'll need for extreme performance actually 
isn't hard to figure out - you need enough free RAM for the OS to cache 
the maximum amount of disk space all your indexes will ever use. 
Normally this will be twice the size of all the indexes on the machine, 
because that's how much disk space will likely be used in a worst-case 
merge scenario (optimize).  That's very expensive, so it is cheaper to 
budget for only the size of the index.

A load of 5000 queries per second is pretty high, and probably something 
you will not achieve with a single-server (not counting backup) 
approach.  All of the tricks that high-volume website developers use are 
also applicable to Solr.

Once you have enough RAM, you need to worry more about the number of 
servers, the number of CPU cores in each server, and the speed of those 
CPU cores.  Testing with actual production queries is the only way to 
find out what you really need.

Beyond hardware design, making the requests as simple as possible and 
taking advantage of caches is important.  Solr has caches for queries, 
filters, and documents.  You can also put a caching proxy (something 
like Varnish) in front of Solr, but that would make NRT updates pretty 
much impossible, and that kind of caching can be difficult to get 
working right.

Thanks,
Shawn


Mime
View raw message