lucene-solr-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Solr Wiki] Update of "SolrCaching" by HossMan
Date Thu, 30 Aug 2012 23:47:33 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.

The "SolrCaching" page has been changed by HossMan:
http://wiki.apache.org/solr/SolrCaching?action=diff&rev1=34&rev2=35

Comment:
more notes on when/if to disable caching, make autowarming it's own wubsection

  
  
  = Tradeoffs =
+ 
+ == Cache Warming and Autowarming ==
+ 
+ If the current Index Searcher is serving requests and when a new searcher is opened, the
new one is 'warmed' while the current one is serving external requests. When the new one is
ready, it is registered so it can serve any new requests while the original one first finishes
the requests it is handling.
+ 
  Increasing autoWarming values will cause additional latency due to auto-warming from the
time that you request a new searcher to be opened until the time that it becomes "registered".
  
  == Caching and Distribution/Replication ==
+ 
- Distribution/Replication gives you a 'new' index on the slave. When Solr is told to use
the new index, the old caches have to be discarded along with the old Index Searcher. That's
when autowarming occurs.
+ Distribution/Replication gives you a 'new' index on the slave. When Solr is told to use
the new index, the old caches have to be discarded along with the old Index Searcher. That's
when warming occurs.
  
- If the current Index Searcher is serving requests and when a new searcher is opened, the
new one is 'warmed' while the current one is serving external requests. When the new one is
ready, it is registered so it can serve any new requests while the original one first finishes
the requests it is handling.
+ == Cache Sizing ==
+ 
+ Beware of the trap of making your cache sizes larger and larger just because you can get
more hits in the cache.  Caches that are too large can often be part of the reason why full
garbage collections happen during or soon after commits.  Sometimes a smaller cache size will
help avoid full garbage collections at the cost of more evictions.  Load testing should be
used to help determine proper cache sizes throughout the searching/indexing lifecycle.
  
  == Disabling Caching ==
- Caching helps only if you are hitting cached objects more than once. If that is not the
case the system is wasting cycles and memory, and you might consider disabling caching by
commenting-out the caching sections in your [[SolrConfigXml|solrconfig.xml]].
  
+ Caching helps only if you are using cached objects more than once.  If that is not the case
the system is wasting cycles and memory, and you might consider disabling caching by commenting-out
the caching sections in your [[SolrConfigXml|solrconfig.xml]].  It's important to remember
however that re-using cached objects isn't just something that happens across multiple requests
-- caches can also be useful when individual requests ask for the same object (ie: faceting
on a field that you also let users "drill down" and filter on.  Before disabling caching completley,
it's usually worth while to first consider:
- == Cache Sizing ==
- Beware of the trap of making your cache sizes larger and larger just because you can get
more hits in the cache.  Caches that are too large can often be part of the reason why full
garbage collections happen during or soon after commits.  Sometimes a smaller cache size will
help avoid full garbage collections at the cost of more evictions.  Load testing should be
used to help determine proper cache sizes throughout the searching/indexing lifecycle.
  
+  * Disabling or reducing the amount of cache warming
+  * Reducing the size of the caches
+ 
+ The right choice may vary for each cache, and should be based on the hit ratios you see
when looking at cache stats as well as your ultimate priorities of query speed, vs warming
time, vs memory.
+ 

Mime
View raw message