lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damien Dudognon <damien.dudog...@ebuzzing.com>
Subject Replication lag after cache optimizations
Date Fri, 31 Aug 2012 14:02:37 GMT
Hi,

We get some troubles with the solr replication after cache optimizations. We use a lot the
facet features.

We have increased the cache size and its initial size. We have also change the queryResultCache
from LRU to FastLRU and the fieldValueCache was activated (see below the detailed configuration).
This optimization step allows to  divide the average time per request by 50 (from 260ms to
5ms).

However, with these modifications we noticed an important replication lag. This issue is not
troublesome for smaller indexes (about 300.000 elements - 3Gb), but it becomes critical if
the index size is significant (30 million elements - 70Gb). In fact, the slaves can't make
up for lag and they become out-of-date (and consequently unusable). 

Has anyone ever been faced this kind of problems?

Our environment :
- Solr 3.4.0
- Java 1.6.0_26
- Debian 6.0.3

Best regards,
Damien

------------------------------------------------------------------
My previous cache settings (fieldValueCache was disabled):
------------------------------------------------------------------
<filterCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0" />
<queryResultCache class="solr.LRUCache" size="512" initialSize="512" autowarmCount="0"
/>
<documentCache class="solr.FastLRUCache" size="512" initialSize="512" autowarmCount="0"
/>
<!--
<fieldValueCache class="solr.FastLRUCache" size="512" autowarmCount="128" showItems="32"
/>
-->

------------------------------------------------------------------
The settings now used:
------------------------------------------------------------------
<filterCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
<queryResultCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
<documentCache class="solr.FastLRUCache" size="16384" initialSize="4096" autowarmCount="0"
/>
<fieldValueCache class="solr.FastLRUCache" size="16384" autowarmCount="1024" showItems="32"
/>

------------------------------------------------------------------
The replication config:
------------------------------------------------------------------
  <requestHandler name="/replication" class="solr.ReplicationHandler" >
    <lst name="master">
      <str name="enable">${solr.enable.master:false}</str>
      <str name="replicateAfter">commit</str>
      <str name="replicateAfter">startup</str>
      <str name="confFiles">schema.xml,stopwords.txt</str>
    </lst>
    <lst name="slave">
      <str name="enable">${solr.enable.slave:false}</str>
      <str name="masterUrl">http://solrmaster:${jetty.port:8083}/solr/en/replication</str>
      <str name="pollInterval">00:00:01</str>
    </lst>
  </requestHandler>
Mime
View raw message