Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Message-Id: <1438864402.342925.349350361.506F47AE@webmail.messagingengine.com>
From: Upayavira <uv@odoko.co.uk>
To: solr-user@lucene.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain
In-Reply-To: <1438861080.21228.11.camel@te-prime>
References: <55C33E5F.6070304@uni-bielefeld.de>
 <1438861080.21228.11.camel@te-prime>
Subject: Re: are facets or MatchAllDocsQuery not cached?
Date: Thu, 06 Aug 2015 13:33:22 +0100

Typically such performance issues with faceting are to do with the time
spend uninverting the index before calculating the facet counts. 

If you indexed the fields with docValues enabled, perhaps you could then
use them for faceting, which might improve performance.

If you are using a non-docValues field, and the second query is faster,
then you could add the query to your static warming, look for
newSearcher in your solrconfig.xml. That will execute your query,
warming the caches used by faceting, before a new searcher is made
available for searches. 

Upayavira

On Thu, Aug 6, 2015, at 12:38 PM, Toke Eskildsen wrote:
> On Thu, 2015-08-06 at 13:00 +0200, Bernd Fehling wrote:
> > Single Index Solr 4.10.4, optimized Index, 76M docs, 235GB index size.
> >
> > I was analysing my solr logs and it turned out that I have some queries
> > which are above 30 seconds qtime while normally the qtime is below 1 second.
> > Looking closer about the queries it turned out that this is for MatchAllDocsQuery(*:*).
> > Next was to turn debugQuery on and see where the bottleneck is.
> > The result was that the facetting is consuming most of the qtime.
> >
> > So the question is, are facets or is facetting not cached?
> 
> As far as I know it is not. 35 seconds for a match-all faceting sounds
> fairly on par with what we are seeing (250M docs, 900GB shard).
> 
> Of course response time is very depending on the field itself. If you
> have very few unique values in your facet field(s), you might try
> facet.method=enum. If that is not the case, your best bet would probably
> be to cache the match-all outside of Solr.
> 
> > My assumption is that the queryResultCache is catching such a MatchAllDocsQuery(*:*).
> 
> It only stores the docIDs.
> 
> I don't know why there is is no all_parameters -> complete_response
> cache in Solr.
> 
> - Toke Eskildsen, State and University Library, Denmark
> 
>