Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of erickerickson@gmail.com
 designates 209.85.220.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <66830ebd69374033b875b822ea0bbfac@CO1PR04MB444.namprd04.prod.outlook.com>
References: 
 <66830ebd69374033b875b822ea0bbfac@CO1PR04MB444.namprd04.prod.outlook.com>
Date: Tue, 15 Apr 2014 19:21:07 -0400
Message-ID: 
 <CAN4YXvcZZx+byZ-cTt_7Cr1Jc-jONU4gRFXN6LsffneeXQsUzA@mail.gmail.com>
Subject: Re: cache warming questions
From: Erick Erickson <erickerickson@gmail.com>
To: solr-user@lucene.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

bq: What does it mean that items will be regenerated or prepopulated
from the current searcher's cache...

You're right, the values aren't cached. They can't be since the
internal Lucene document id is used to identify docs, and due to
merging the internal ID may bear no relation to the old internal ID
for a particular document.

I find it useful to think of Solr's caches as a  map where the key is
the "query" and the value is some representation of the found
documents. The details of the value don't matter, so I'll skip them.

What matters is the key. Consider the filter cache. You put something
like &fq=3Dprice:[0 TO 100] on a URL. Solr then uses the fq  clause as
the key to the filterCache.

Here's the sneaky bit. When you specify an autowarm count of N for the
filterCache, when a new searcher is opened the first N keys from the
map are re-executed in the new searcher's context and the results put
into the new searcher's filterCache.

bq:  ...how does auto warming and explicit warming work together?

They're orthogonal. IOW, the autowarming for each cache is executed as
well as the newSearcher static warming queries. Use the static queries
to do things like fill the sort caches etc.

Incidentally, this bears on why there's a "firstSearcher" and
"newSearcher". The newSearcher queries are run in addition to the
cache autowarms. firstSearcher static queries are only run when a Solr
server is started the first time, and there are no cache entries to
autowarm. So the firstSearcher queries might be quite a bit more
complex than newSearcher queries.

HTH,
Erick

On Tue, Apr 15, 2014 at 1:55 PM, Matt Kuiper <matt.kuiper@issinc.com> wrote=
:
> Hello,
>
> I have a few questions regarding how Solr caches are warmed.
>
> My understanding is that there are two ways to warm internal Solr caches =
(only one way for document cache and lucene FieldCache):
>
> Auto warming - occurs when there is a current searcher handling requests =
and new searcher is being prepared.  "When a new searcher is opened, its ca=
ches may be prepopulated or "autowarmed" with cached object from caches in =
the old searcher. autowarmCount is the number of cached items that will be =
regenerated in the new searcher."    http://wiki.apache.org/solr/SolrCachin=
g#autowarmCount
>
> Explicit warming - where the static warming queries specified in Solrconf=
ig.xml for newSearcher and firstSearcher listeners are executed when a new =
searcher is being prepared.
>
> What does it mean that items will be regenerated or prepopulated from the=
 current searcher's cache to the new searcher's cache?  I doubt it means co=
py, as the index has likely changed with a commit and possibly invalidated =
some contents of the cache.  Are the queries, or filters, that define the c=
ontents of the current caches re-executed for the new searcher's caches?
>
> For the case where auto warming is configured, a current searcher is acti=
ve, and static warming queries are defined how does auto warming and explic=
it warming work together? Or do they?  Is only one type of warming activate=
d to fill the caches?
>
> Thanks,
> Matt