Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of mkhludnev@griddynamics.com
 designates 209.85.216.43 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <alpine.DEB.2.02.1410011011150.1884@frisbee>
References: <46AD7E4C-23A5-4261-A1EE-5AD52A2AA663@transpac.com>
 <52613889.6040200@kelkoo.com>
 <CAGe-MLJ+6y1at+OunK3sGaCFF6zGtJq_Nin9_3SHN0kFuqXkBA@mail.gmail.com>
 <C5EA484A-1998-4AAE-AF0D-8B2DE2E81BB1@flax.co.uk>
 <CANGii8fNtF3rOZVZ1DWrgw1Yk4s0K-9UGr2Qxup0MRGPwnd5uw@mail.gmail.com>
 <96B287E9-E4BD-45DB-848D-BC634BB6880B@flax.co.uk>
 <CAN4YXvf+Rbtj7fbaO+97dSPZjB+Y39VCuQZGKYoXEYUdF2q_AA@mail.gmail.com>
 <alpine.DEB.2.02.1410011011150.1884@frisbee>
From: Mikhail Khludnev <mkhludnev@griddynamics.com>
Date: Wed, 1 Oct 2014 23:27:41 +0400
Message-ID: 
 <CANGii8euQD0q6sTuE3BOBQemedHA8Zgtyuqx1P-OPCCUmS2wDw@mail.gmail.com>
Subject: Re: Filter cache pollution during sharded edismax queries
To: solr-user <solr-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=089e0122f1f6c958390504618055

--089e0122f1f6c958390504618055
Content-Type: text/plain; charset=UTF-8

Hoss,

Nice to hear you! I wonder if there is a sequence chart, or maybe a deck,
which explains the whole picture of distributed search, especially these
ones?
If it hasn't been presented to community so far, I'm aware of one
conference which can accept such talk. WDYT?

On Wed, Oct 1, 2014 at 9:17 PM, Chris Hostetter <hossman_lucene@fucit.org>
wrote:

>
> : +1 for using a different cache, but that's being quite unfamiliar with
> the
> : code.
>
> in (a) common case, people tend to "drill down" and filter on facet
> constraints -- so using a special purpose cache for the refinements would
> result in redundent caching of the same info in multiple places.
>
> : > > What's the point to refine these counts? I've thought that it make
> sense
> : > > only for facet.limit ed requests. Is it correct statement? can those
> who
>
> refinement only happens if facet.limit is used and there are eligable
> "top" constraints that were not returned by some shards.
>
> : > > suffer from the low performance, just unlimit  facet.limit to avoid
> that
> : > > distributed hop?
>
> As noted, setting facet.limit=-1 might help for low cardinality fields to
> ensure that every shard returns a count for every value and no-refinement
> is needed, but that doesn't really help you for fields with
> unknown/unbounded cardinality.
>
> As part of the distributed pivot faceting work, the amount of
> "overrequest" done in phase 1 (for both facet.pivot & facet.field) was
> made configurable via 2 new parameters...
>
>
> https://lucene.apache.org/solr/4_10_0/solr-solrj/org/apache/solr/common/params/FacetParams.html#FACET_OVERREQUEST_RATIO
>
> https://lucene.apache.org/solr/4_10_0/solr-solrj/org/apache/solr/common/params/FacetParams.html#FACET_OVERREQUEST_COUNT
>
> ...so depending on the distribution of your data, you might find that by
> adjusting those values to increase the amount of overrequesting done, you
> can decrease the amount of refinement needed -- but there are obviously
> tradeoffs.
>
>
>
> -Hoss
> http://www.lucidworks.com/
>


-- 
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics

<http://www.griddynamics.com>
<mkhludnev@griddynamics.com>

--089e0122f1f6c958390504618055--