lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tushar_k47 <masterofpuppetss...@gmail.com>
Subject Solr Out Of Memory with Field Collapsing
Date Mon, 29 Jul 2013 15:14:29 GMT
Hi,

We are using Field collapsing feature with multiple shards. We ran into into
Out of Memory errors on one of the shards. We use filed collapsing on a
particular field which has only one specific value on the shard that goes
out of memory. Interestingly the Out of Memory error recurred multiple times
during the day (about 4 times in 24 hours) without any significant deviation
in traffic from the normal or the nature of queries being run. The max heap
size allocated to the shard 8 Gb.

Since then we have done the following and the problem seems to be arrested
for now -

1. Added more horizontal slaves, from 3 we have brought this up to 6.
2. We have increased the replication poll interval from 5 minutes to 20
minutes.
3. We have decreased the minimum heap allocation to this tomcat to 1Gb.
Earlier this was 4Gb.

The typical size of index directory on the problem shard is around 1Gb,
about 1 million documents in all. The average requests served are about
10/second for this shard. 

We have tried replaying the entire logs for the day on a test environment
but somehow it never goes out of memory with the same heap settings. Now we
are not certain that this would not happen again. 

Can someone suggest what could be the problem here ? Any help would be
greatly appreciated.

Regards,
Tushar

 



--
View this message in context: http://lucene.472066.n3.nabble.com/Solr-Out-Of-Memory-with-Field-Collapsing-tp4081076.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message