lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Hull <char...@flax.co.uk>
Subject Re: Facet queries blow out the filterCache
Date Fri, 02 Oct 2015 08:23:31 GMT
On 01/10/2015 23:31, Jeff Wartes wrote:
> It still inserts if I address the core directly and use distrib=false.
>
> I’ve got a few collections sharing the same config, so it’s surprisingly
> annoying to
> change solrconfig.xml right now, but it seemed pretty clear the query is
> the thing being cached, since
> the cache size only changes when the query does.

Hi Jeff,

I think you may be hitting the same issue we found:
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201409.mbox/%3CCAGe-MLJ+6y1at+OunK3sGaCFF6zGtJq_Nin9_3SHN0kFuqXkBA@mail.gmail.com%3E

Distributed faceting uses the filter cache, where you wouldn't expect it 
to. The solution was to set facet.limit to -1.

Best

Charlie
>
>
>
> On 10/1/15, 3:01 PM, "Mikhail Khludnev" <mkhludnev@griddynamics.com> wrote:
>
>> hm..
>> This option was useful for introspecting cache content
>> https://wiki.apache.org/solr/SolrCaching#showItems It might help you to
>> find-out a cause.
>> I'm still blaming distributed requests, it expained here
>> https://cwiki.apache.org/confluence/display/solr/Faceting#Faceting-Over-Re
>> questParameters
>> eg does it happen if you run with distrib=false?
>>
>> On Fri, Oct 2, 2015 at 12:27 AM, Jeff Wartes <jwartes@whitepages.com>
>> wrote:
>>
>>>
>>> No change, still shows an insert per-request. As does a simplified
>>> request
>>> with only the facet params
>>> "&facet.field=city&facet=true"
>>>
>> by default it's 100
>> https://cwiki.apache.org/confluence/display/solr/Faceting#Faceting-Theface
>> t.limitParameter
>> and can cause filtering by values, it can be seen in logs, btw.
>>
>>>
>>> It’s definitely facet related though, facet=false eliminates the insert.
>>>
>>>
>>>
>>> On 10/1/15, 1:50 PM, "Mikhail Khludnev" <mkhludnev@griddynamics.com>
>>> wrote:
>>>
>>>> what if you set f.city.facet.limit=-1 ?
>>>>
>>>> On Thu, Oct 1, 2015 at 7:43 PM, Jeff Wartes <jwartes@whitepages.com>
>>>> wrote:
>>>>
>>>>>
>>>>> I’m doing some fairly simple facet queries in a two-shard 5.3
>>> SolrCloud
>>>>> index on fields like this:
>>>>>
>>>>> <field name="city" type="string" indexed="true" stored="false"
>>>>> docValues="true”/>
>>>>>
>>>>> that look something like this:
>>>>>
>>>
>>>>> q=...&fl=id,score&facet.field=city&facet=true&f.city.facet.mincount=1&f
>>>>> .c
>>>>> it
>>>>> y.facet.limit=50&rows=0&start=0&facet.method=fc
>>>>>
>>>>> (no, NOT facet.method=enum - the usage of the filterCache there is
>>>>> pretty
>>>>> well documented)
>>>>>
>>>>> Watching the filterCache stats, it appears that every one of these
>>>>> queries
>>>>> causes the "inserts" counter to be incremented by one. Distinct "q="
>>>>> queries also increase the "size", and eviction happens as normal. If
>>> I
>>>>> repeat the same query a few times, "lookups" is not incremented, so
>>>>> these
>>>>> entries generally appear to be completely wasted. (Although when
>>>>> running a
>>>>> lot of these queries, it appears as though a very small set also
>>>>> increment
>>>>> the "lookups" counter, but only a small set, and I haven’t figured
>>> out
>>>>> why
>>>>> some are special.)
>>>>>
>>>>> So the question is, why does this facet query have anything to do
>>> with
>>>>> the
>>>>> filterCache? This causes a huge amount of filterCache churn with no
>>>>> apparent benefit.
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Sincerely yours
>>>> Mikhail Khludnev
>>>> Principal Engineer,
>>>> Grid Dynamics
>>>>
>>>> <http://www.griddynamics.com>
>>>> <mkhludnev@griddynamics.com>
>>>
>>>
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>> Principal Engineer,
>> Grid Dynamics
>>
>> <http://www.griddynamics.com>
>> <mkhludnev@griddynamics.com>
>


-- 
Charlie Hull
Flax - Open Source Enterprise Search

tel/fax: +44 (0)8700 118334
mobile:  +44 (0)7767 825828
web: www.flax.co.uk

Mime
View raw message