lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Solr 5.2.1 versus Solr 4.7.0 performance
Date Thu, 27 Aug 2015 14:44:49 GMT
On 8/27/2015 3:26 AM, Toke Eskildsen wrote:
> On Thu, 2015-08-27 at 11:23 +0300, Esther Goldbraich wrote:
>> We are using GC tuning options: Xgcpolicy:gencon , verbose:gc.
>> RAM: 64GB
>> Solr heap: -Xms512m -Xmx32768m
>> Index per server: 500G

Is this in use for Solr 5.2.1 as well?  The start script for Solr 5.x
includes garbage collection tuning that has worked well for most people.

I have no idea what the "gencon" strategy actually does, and I cannot
find much information with Google.  Most of the info I can find on this
setting indicates this setting on Oracle Java is for JRockit, which is
gone in Java 7.  I suspect that it doesn't do much beyond changing which
collectors are in use.  This is *not* enough GC tuning for Solr, and if
you are running Oracle Java 7 or OpenDK 7, may not actually do anything
useful at all.  It looks like it is a valid setting on IBM Java, but
there are plenty of reasons (bugs) to never use Java from IBM.

The GC settings in the bin/solr start script included with 5.x are good,
or you can look at my wiki page for info on all the GC testing that I
have done personally:

https://wiki.apache.org/solr/ShawnHeisey

> Expecting "Your RAM size should be equal to index size"-posts to arrive
> in 3, 2, 1...

I see that I am predictable. :)

There is about 32GB left over after Java is done allocating memory for
Solr, assuming there isn't anything else on the system that requires a
significant chunk of memory, like a webserver or a database server.

Like Toke mentions, I will tell you that for *ideal* performance, your
available RAM should be equal to the index size.  In a perfect world,
you would have at least 532GB of RAM -- 32GB for Java and 500GB for
caching the index.  Since that's not a typical RAM size for a server,
call it 576GB or 640GB.

Perfection is not required for Solr, but I can tell you that 32GB of RAM
for 500GB of index will almost certainly lead to performance issues.
This most likely wouldn't be enough even if your index is on SSD.

If I couldn't achieve the dream of 512GB to run 500GB of index, I would
hope for at least 128GB of RAM for SSD storage or at least 256GB of RAM
for spinning disks, and I would not be surprised to learn that's not
enough either.

I suspect this may be the majority of why you don't see a performance
improvement with the newer Java version.  Newer versions generally have
better performance, but only if your hardware has enough resources for
the job.

Thanks,
Shawn


Mime
View raw message