lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Is there any performance cost of using lots of OR in the solr query
Date Fri, 06 Apr 2012 13:00:26 GMT
On 4/5/2012 3:49 PM, Erick Erickson wrote:
> Of course putting more clauses in an OR query will
> have a performance cost, there's more work to do....
>
> OK, being a smart-alec aside you will probably
> be fine with a few hundred clauses. The question
> is simply whether the performance hit is acceptable.
> I'm afraid that question can't be answered in the
> abstract, you'll have to test...
>
> Since you're putting them in an fq, there's also some chance
> that they'll be re-used from the cache, at least if there
> are common patterns.

Roz,

I have a similar situation going on in my index.  Because employees have 
access to far more than real users, they get filter queries constructed 
that have HUGE number of clauses in them.  We have implemented a new 
field for a feature that we call "search groups" but it has not 
penetrated all aspects of the application yet.  Also, until we can make 
those groups use a hierarchy, which is not a trivial undertaking, we may 
be stuck with large filter queries.

These complex filters have led to a problem that you have probably not 
considered - really long filterCache autowarm times.  I have reduced the 
autoWarm value on my filterCache to FOUR, and there are still times that 
the autowarm takes up to 60 seconds.  Most of the time it is only a few 
seconds, with up to 30 seconds being relatively common.

I just thought of a new localparam feature for this situation and filed 
SOLR-3333.  I will talk to our developers about using the existing 
localparam that skips filterCache entirely.

Thanks,
Shawn


Mime
View raw message