lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: cache warming questions
Date Tue, 15 Apr 2014 23:21:07 GMT
bq: What does it mean that items will be regenerated or prepopulated
from the current searcher's cache...

You're right, the values aren't cached. They can't be since the
internal Lucene document id is used to identify docs, and due to
merging the internal ID may bear no relation to the old internal ID
for a particular document.

I find it useful to think of Solr's caches as a  map where the key is
the "query" and the value is some representation of the found
documents. The details of the value don't matter, so I'll skip them.

What matters is the key. Consider the filter cache. You put something
like &fq=price:[0 TO 100] on a URL. Solr then uses the fq  clause as
the key to the filterCache.

Here's the sneaky bit. When you specify an autowarm count of N for the
filterCache, when a new searcher is opened the first N keys from the
map are re-executed in the new searcher's context and the results put
into the new searcher's filterCache.

bq:  ...how does auto warming and explicit warming work together?

They're orthogonal. IOW, the autowarming for each cache is executed as
well as the newSearcher static warming queries. Use the static queries
to do things like fill the sort caches etc.

Incidentally, this bears on why there's a "firstSearcher" and
"newSearcher". The newSearcher queries are run in addition to the
cache autowarms. firstSearcher static queries are only run when a Solr
server is started the first time, and there are no cache entries to
autowarm. So the firstSearcher queries might be quite a bit more
complex than newSearcher queries.

HTH,
Erick

On Tue, Apr 15, 2014 at 1:55 PM, Matt Kuiper <matt.kuiper@issinc.com> wrote:
> Hello,
>
> I have a few questions regarding how Solr caches are warmed.
>
> My understanding is that there are two ways to warm internal Solr caches (only one way
for document cache and lucene FieldCache):
>
> Auto warming - occurs when there is a current searcher handling requests and new searcher
is being prepared.  "When a new searcher is opened, its caches may be prepopulated or "autowarmed"
with cached object from caches in the old searcher. autowarmCount is the number of cached
items that will be regenerated in the new searcher."    http://wiki.apache.org/solr/SolrCaching#autowarmCount
>
> Explicit warming - where the static warming queries specified in Solrconfig.xml for newSearcher
and firstSearcher listeners are executed when a new searcher is being prepared.
>
> What does it mean that items will be regenerated or prepopulated from the current searcher's
cache to the new searcher's cache?  I doubt it means copy, as the index has likely changed
with a commit and possibly invalidated some contents of the cache.  Are the queries, or filters,
that define the contents of the current caches re-executed for the new searcher's caches?
>
> For the case where auto warming is configured, a current searcher is active, and static
warming queries are defined how does auto warming and explicit warming work together? Or do
they?  Is only one type of warming activated to fill the caches?
>
> Thanks,
> Matt

Mime
View raw message