Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of apache@elyograg.org
 designates 166.70.79.219 as permitted sender)
Message-ID: <54906703.8040206@elyograg.org>
Date: Tue, 16 Dec 2014 10:08:19 -0700
From: Shawn Heisey <apache@elyograg.org>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: solr-user@lucene.apache.org
Subject: Re: OutOfMemoryError
References: 
 <CA+US4SsXXxns6iST0dCWGsQ7Juv9YvcX_tiVRtaFYzPbmoZ5YQ@mail.gmail.com>
In-Reply-To: 
 <CA+US4SsXXxns6iST0dCWGsQ7Juv9YvcX_tiVRtaFYzPbmoZ5YQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

On 12/16/2014 9:55 AM, Trilok Prithvi wrote:
> We are getting OOME pretty often (every hour or so). We are restarting
> nodes to keep up with it.
>
> Here is our setup:
> SolrCloud 4.10.2 (2 shards, 2 replicas) with 3 zookeepers.
>
> Each node has:
> 16GB RAM
> 2GB JVM (Xmx 2048, Xms 1024)
> ~100 Million documents (split among 2 shards - ~50M on each shard)
> Solr Core is about ~16GB of data on each node.
>
> *Physical Memory is almost always 99% full.*

I'm pretty sure that a 2GB heap will simply not be big enough for 100 
million documents.  The fact that you can get it to function for even an 
hour is pretty amazing.

If you can upgrade the memory beyond 16GB, you should ... and you'll 
need to increase your Java heap.  I would use 4GB as a starting point.

http://wiki.apache.org/solr/SolrPerformanceProblems#How_much_heap_space_do_I_need.3F

It's completely normal for physical memory to be full.  The OS uses 
available memory for disk caching.

http://en.wikipedia.org/wiki/Page_cache

Thanks,
Shawn