lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Replication lag after cache optimizations
Date Wed, 05 Sep 2012 00:01:11 GMT

: However, with these modifications we noticed an important replication 

I'm not sure how exactly you are measuring/defining "replication lag" but 
if you mean "lag in how long until the newly replicated documents are 
visible in searches" then that _may_ be fairly easy to explain...

: My previous cache settings (fieldValueCache was disabled):

FYI: for historical reasons, there is always a fieldValueCache, even if 
you don't declare one....

https://wiki.apache.org/solr/SolrCaching#fieldValueCache

: <fieldValueCache class="solr.FastLRUCache" size="16384" autowarmCount="1024" showItems="32"
/>

you have gone from using the hardcoded deault fieldValueCache, (which had 
no warming configured at all) to configuring an autowarmCount of 1024 -- 
you should easily be able to see in the logs that the "newSearcher" time 
on your machines is much longer since this change, as it autowarms thouse 
fieldValueCache entries.

This means that, compared to your preivous settings, the "first 
request" that attempts to use those fieldValueCache entires should be 
much faster then before, but the trade off is that you are spending 
the time to generate those cache entires "up-front" before you allow any 
requests to see the updated index at all.  

FWIW: the entries in the fieldValueCache are keyed off of field name (they 
are very bug UnInvertedField objects, and are typtically very few of them 
-- this is why, IIRC, yonik recommends no wautowarming of fieldValueCache 
at all) so having a size of 16384 and an autowarmCount of 1024 is probably 
overkill ... i suspect if you check the actual size at runtime you'll see 
that there are way fewer entires then that -- if you have anywhere close 
to 16384 entires i would love to hear more about your usecase.


-Hoss

Mime
View raw message