Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of scottblanc@gmail.com designates
 209.85.212.191 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=A0Rx324zQHsR61//Z9eedeLO5XsLRbiYbaEdTzMfpJM292wh/31C2R1wqu/gMdPOnN
         TaWHYzojEnlHccaLtFOahWu5hduKULMHoOamB58fx7RMmjFANSi4Bx1QOLNtP7Po12kc
         KSp7PO8Q62RFh+bwEp8Iv0Aa7SQjAS/+unNQM=
MIME-Version: 1.0
In-Reply-To: <D3EAEADE-DF83-4E65-BBC9-34A9D80E8B73@apache.org>
References: <401b577e0910080754o52c72a7kdd4fdc205753e3f2@mail.gmail.com>
	 <401b577e0910080756m2f36ae8bo7c573465376d2df6@mail.gmail.com>
	 <85635AEF-DEBE-49CF-B04A-CC45CB4B7CF6@apache.org>
	 <D3EAEADE-DF83-4E65-BBC9-34A9D80E8B73@apache.org>
Date: Fri, 9 Oct 2009 14:19:46 -0700
Message-ID: <401b577e0910091419pa43264bs8236bbda8daab4fa@mail.gmail.com>
Subject: Re: Question about how to speed up custom scoring
From: scott w <scottblanc@gmail.com>
To: java-user@lucene.apache.org
Content-Type: multipart/alternative; boundary=001485e8ad58b0990a047587234e

--001485e8ad58b0990a047587234e
Content-Type: text/plain; charset=ISO-8859-1

Right exactly. I looked into payload initially and realized it wouldn't work
for my use case.

On Fri, Oct 9, 2009 at 2:00 PM, Grant Ingersoll <gsingers@apache.org> wrote:

> Oops, just reread and realized you wanted query time weights.  Payloads are
> an index time thing.
>
>
> On Oct 9, 2009, at 5:49 PM, Grant Ingersoll wrote:
>
>  If you are trying to add specific term weights to terms in the index and
>> then incorporate them into scoring, you might benefit from payloads and the
>> PayloadTermQuery option.  See
>> http://www.lucidimagination.com/blog/2009/08/05/getting-started-with-payloads/
>>
>> -Grant
>>
>> On Oct 8, 2009, at 11:56 AM, scott w wrote:
>>
>>  Oops, forgot to include the class I mentioned. Here it is:
>>>
>>> public class QueryTermBoostingQuery extends CustomScoreQuery {
>>> private Map<String, Float> queryTermWeights;
>>> private float bias;
>>> private IndexReader indexReader;
>>>
>>> public QueryTermBoostingQuery( Query q, Map<String, Float> termWeights,
>>> IndexReader indexReader, float bias) {
>>>  super( q );
>>>  this.indexReader = indexReader;
>>>  if (bias < 0 || bias > 1) {
>>>    throw new IllegalArgumentException( "Bias must be between 0 and 1" );
>>>  }
>>>  this.bias = bias;
>>>  queryTermWeights = termWeights;
>>> }
>>>
>>> @Override
>>> public float customScore( int doc, float subQueryScore, float valSrcScore
>>> ) {
>>>  Document document;
>>>  try {
>>>    document = indexReader.document( doc );
>>>  } catch (IOException e) {
>>>    throw new SearchException( e );
>>>  }
>>>  float termWeightedScore = 0;
>>>
>>>  for (String field : queryTermWeights.keySet()) {
>>>    String docFieldValue = document.get( field );
>>>    if (docFieldValue != null) {
>>>      Float weight = queryTermWeights.get( field );
>>>      if (weight != null) {
>>>        termWeightedScore += weight * Float.parseFloat( docFieldValue );
>>>      }
>>>    }
>>>  }
>>>  return bias * subQueryScore + (1 - bias) * termWeightedScore;
>>> }
>>> }
>>>
>>> On Thu, Oct 8, 2009 at 7:54 AM, scott w <scottblanc@gmail.com> wrote:
>>>
>>>  I am trying to come up with a performant query that will allow me to use
>>>> a
>>>> custom score where the custom score is a sum-product over a set of query
>>>> time weights where each weight gets applied only if the query time term
>>>> exists in the document . So for example if I have a doc with three
>>>> fields:
>>>> company=Microsoft, city=Redmond, and size=large, I may want to score
>>>> that
>>>> document according to the following function: city==Microsoft ? .3 : 0 *
>>>> size ==large ? 0.5 : 0 to get a score of 0.8. Attached is a subclass I
>>>> have
>>>> tested that implements this with one extra component which is that it
>>>> allow
>>>> the relevance score to be combined in.
>>>>
>>>> The problem is this custom score is not performant at all. For example,
>>>> on
>>>> a small index of 5 million documents with 10 weights passed in it does
>>>> 0.01
>>>> req/sec.
>>>>
>>>> Are there ways to make to compute the same custom score but in a much
>>>> more
>>>> performant way?
>>>>
>>>> thanks,
>>>> Scott
>>>>
>>>>
>> --------------------------
>> Grant Ingersoll
>> http://www.lucidimagination.com/
>>
>> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
>> Solr/Lucene:
>> http://www.lucidimagination.com/search
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
> --------------------------
> Grant Ingersoll
> http://www.lucidimagination.com/
>
> Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using
> Solr/Lucene:
> http://www.lucidimagination.com/search
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

--001485e8ad58b0990a047587234e--