lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Higher Memory Usage with solr 4.4
Date Wed, 11 Sep 2013 16:36:05 GMT
On 9/11/2013 8:54 AM, Kuchekar wrote:
>       We are using solr 4.4 on Linux with OpenJDK 64-Bit. We started the
> Solr with 40GB but we noticed that the QTime is way high compared to
> similar on 3.5 solr.
> Both the 3.5 and 4.4 solr's configurations and schema are similarly
> constructed. Also during the triage we found the physical memory to be
> utilized at 95..%.

A 40GB heap is *huge*.  Unless you are dealing with millions of 
super-large documents or many many millions of smaller documents, there 
should be no need for a heap that large.  Additionally, if you are 
allocating most of your system memory to Java, then you will have little 
or no RAM available for OS disk caching, which will cause major 
performance issues.

For most indexes, memory usage should be less after an upgrade, but 
there are exceptions.

I see that you had an earlier question about stored field compression, 
and that you talked about exporting data from your 3.5 install to index 
into 4.4, in which you had stored every field, including copyFields.

If you have a lot of stored data, memory usage for decompression can 
become a problem.  It's usually a lot better to store minimal 
information, just enough to display a result grid/list, and some ID 
information so that when someone clicks on an individual result, you can 
retrieve the entire record from another data source, like a database or 
a filesystem.

Here's a more exhaustive list of potential performance and memory 
problems with Solr:

http://wiki.apache.org/solr/SolrPerformanceProblems

OpenJDK may be problematic, especially if it's version 6.  With Java 7, 
OpenJDK is actually the reference implementation, so if you are using 
OpenJDK 7, I would be less concerned.  With either version, Oracle Java 
tends to produce better results.

Thanks,
Shawn


Mime
View raw message