lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rogalon <nico.beche...@me.com>
Subject Re: Function Query performance in combination with filters
Date Wed, 17 Apr 2013 07:45:09 GMT
Rogalon wrote
> Am 16. April 2013 um 14:46 schrieb "Yonik Seeley-4 [via Lucene]" &lt;

> ml-node+s472066n4056299h21@.nabble

> &gt;:
> 
>> On Tue, Apr 16, 2013 at 7:51 AM, Rogalon <[hidden email]> wrote:
>>
>> > Hi,
>> > I am using pretty complex function queries to completely customize (not
>> only
>> > boost) the score of my result documents that are retrieved from an
>> index of
>> > approx 10e7 documents. To get to an acceptable level of performance I
>> > combine my query with filters in the following way (very short
>> example):
>> >
>> >
>> q=_val_:"sum(termfreq(fieldname,`word`),termfreq(fieldname2,`word2`))"&fq=fieldname:`word`&fq=fieldname2:`word2`
>> >
>> > Although I always have (because of the filter) approx 50.000 docs in
>> the
>> > result set, the search times vary (depending on the actual query)
>> between
>> > 100ms and 6000ms.
>> >
>> > My understanding was that the scoring function is only applied to the
>> result
>> > set from the filters.
>>
>> That should be the case.
>>
>> > But based on what I am seeing it seems that a lot more
>> > documents are actually put through the _val_ function.
>>
>> How did you verify this? 
> 
> Thanks for taking a look at my problem.
> 
> For now - I verified just by taking a look at the query times and doing
> some simple experiments.
> 
> If I am not using the function query at all (q=*:*&fq=...), the approx.
> 50.000 results from the filters are always returned within 200-300ms. This
> is pretty stable. If I have a (test) index of 50.000 documents (instead of
> the the 10e7 index) only and I pass every document through the _val_ query
> (without any filters), this takes about 150ms which in my case would be
> ok.
> 
> Applying no filters to the function query on the 10e7 index leads to
> search times at about 6000ms which is too much.
> 
> But if I use the filters as stated above I get returned 50.000 documents
> but the query times suddenly start to vary between 100ms and 6000ms. Some
> of my filters might actually be on stop words which appear in every other
> document in the index but that seems to really hurt performance only if
> the function query is used.
> 
>      Greetings, Nico     
>>
>>
>> -Yonik
>> http://lucidworks.com
>>
>>
>> If you reply to this email, your message will be added to the discussion
>> below:
>> http://lucene.472066.n3.nabble.com/Function-Query-performance-in-combination-with-filters-tp4056283p4056299.html
>>  
>> To unsubscribe from Function Query performance in combination with
>> filters, click here.
>> NAML
>>

Any idea what else I could try to figure out the issue? Thanks in advance
;-)



--
View this message in context: http://lucene.472066.n3.nabble.com/Function-Query-performance-in-combination-with-filters-tp4056283p4056609.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message