lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Performance gain with setting !cache=false in the query for complex queries
Date Tue, 25 Aug 2015 02:49:43 GMT
Well, It Depends (tm). I've certainly seen response times on that order, it all
revolves around the complexity of the queries, how much faceting you're
doing, all that kind of thing.

If always specifying cache=false works for you, go for it. The only caution I
would add is that randomly generating fq clauses may be skewing your
results. Yes, there's a penalty up front, but like many caching problems once
the caches are filled, things are much faster.

So what I'd do is let it run in production for a while, then cull all
of the searches
out of the log. I'd then copy these to another file and take out all
the cache=false
statements, _then_ run the two sets independently at Solr and compare. In
particular, for the run where you took cache=false out of the queries, what is
the filter cache hit ratio? If it is quite high you may want to go ahead and
turn caching back on. You'll have to play with the cache size to see how much
performance you can squeeze out here.

All that said, if you know a-priori that there is very little
likelihood of getting
a high hit ratio on the filterCache, don't bother ;).

And, make sure you aren't falling into the trap of using bare NOW in your fq
clauses unless you absolutely need to, in which case every fq clause with
NOW should have cache=false. See: the link I pasted before

Best,
Erick


On Mon, Aug 24, 2015 at 12:43 PM, wwang525 <wwang525@gmail.com> wrote:
> Hi Erick,
>
> The earlier test was done through individual requests. However, my load test
> is even better.
>
> (1) load test (3 requests/per second/per core) immediately after restarting
> Solr: average response time: 122 ms
> (2) load test (5 requests/per second/per core) immediately after restarting
> Solr: average response time: 120 ms
>
> (3) warm-up (filter cache not warmed up with !cache=false) with a load of 3
> requests/per second/per core for 40 rounds, then load test with a load of 5
> requests/per second/per core for 40 rounds: average response time: 72 ms
>
> It is now very obvious that the previously slower query response time (on
> average: 500ms) with filter cache enabled in our demanding query was due to
> extra processing to fill the cache for all the randomly generated requests.
>
> This performance (<100 ms) should be good enough in production for our
> project. However, I would like to know if this response time is a typical
> "Solr speed" based on 15 M records.
>
> Thanks
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Performance-gain-with-setting-cache-false-in-the-query-for-complex-queries-tp4224931p4224988.html
> Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message