Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: local policy)
Message-ID: <4DE66567.8030907@jhu.edu>
Date: Wed, 01 Jun 2011 12:14:31 -0400
From: Jonathan Rochkind <rochkind@jhu.edu>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US;
 rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10
MIME-Version: 1.0
To: solr-user@lucene.apache.org
Subject: Re: What's your query result cache's stats?
References: <201105312302.16611.markus.jelsma@openindex.io>
 <4DE662D8.7030008@elyograg.org>
In-Reply-To: <4DE662D8.7030008@elyograg.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I believe you need SOME query cache even with low hit counts, for things 
like a user paging through results. You want the query to still be in 
the cache when they go to the next page or what have you. Other 
operations like this may depend on the query cache too for good 
performance.

So even with a low hit rate, you still want enough query cache that all 
the "current" queries, all the queries someone is in the middle of doing 
something with and may do more with can stay in the cache. (what things 
those are can depend on your particular client interface).   So the 
cache hit count may not actually be a good guide to sizing your query 
cache.

Correct me if I'm wrong, but this is what I've been thinking.

On 6/1/2011 12:03 PM, Shawn Heisey wrote:
> On 5/31/2011 3:02 PM, Markus Jelsma wrote:
>> Hi,
>>
>> I've seen the stats page many times, of quite a few installations and 
>> even
>> more servers. There's one issue that keeps bothering me: the 
>> cumulative hit
>> ratio of the query result cache, it's almost never higher than 50%.
>>
>> What are your stats? How do you deal with it?
>
> Below are my stats.
>
> I will be lowering my warmcounts dramatically when I respin for 3.1.  
> The 28 second warm time is too high for me.  I don't think it's going 
> to make a lot of difference in performance.  I think most of the 
> warming benefit is realized after the first few queries.
>
> queryResultCache:
> Concurrent LRU Cache(maxSize=1024, initialSize=1024, minSize=921, 
> acceptableSize=972, cleanupThread=true, autowarmCount=64, 
> regenerator=org.apache.solr.search.SolrIndexSearcher$3@60c0c8b5)
>
> lookups : 932
> hits : 528
> hitratio : 0.56
> inserts : 403
> evictions : 0
> size : 449
> warmupTime : 28198
> cumulative_lookups : 980357
> cumulative_hits : 622726
> cumulative_hitratio : 0.63
> cumulative_inserts : 369692
> cumulative_evictions : 83711
>
> lookups : 68543
> hits : 57286
> hitratio : 0.83
> inserts : 11357
> evictions : 0
> size : 11357
> warmupTime : 0
> cumulative_lookups : 219118491
> cumulative_hits : 179119106
> cumulative_hitratio : 0.81
> cumulative_inserts : 39999385
> cumulative_evictions : 32833254
>
>
> documentCache:
> LRU Cache(maxSize=16384, initialSize=4096)
>
> lookups : 68543
> hits : 57286
> hitratio : 0.83
> inserts : 11357
> evictions : 0
> size : 11357
> warmupTime : 0
> cumulative_lookups : 219118491
> cumulative_hits : 179119106
> cumulative_hitratio : 0.81
> cumulative_inserts : 39999385
> cumulative_evictions : 32833254
>
>
> filterCache:
> LRU Cache(maxSize=512, initialSize=512, autowarmCount=32, 
> regenerator=org.apache.solr.search.SolrIndexSearcher$2@6910b640)
>
> lookups : 859
> hits : 464
> hitratio : 0.54
> inserts : 465
> evictions : 0
> size : 464
> warmupTime : 27747
> cumulative_lookups : 682600
> cumulative_hits : 355130
> cumulative_hitratio : 0.52
> cumulative_inserts : 327479
> cumulative_evictions : 161624
>
>
>