lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominika Puzio <strz...@wp-sa.pl>
Subject Re: Question about CustomScoreQuery
Date Tue, 21 Feb 2012 12:29:00 GMT
Thanks for the help! I feel this performance advice about FieldCache in
ctor saved me a lot of time :) I've done what you said and it works.

--
Dominika

On 21.02.2012 10:46, Uwe Schindler wrote:
> It looks like you already implemented a CustomScoreProvider.
>
> You are retrieving the FieldCache on every document, which slows down
> immense (it's a sycronized cache lookup). The correct way is:
>
> Override CSQ.getCustomScoreProvider and return your own CSP there. The CSP
> itself should get the FieldCache for the required field in the ctor and
> store it as a class instance field (Lucene will as for a new
> CustomScoreProvider for each index segment).
>
> In CSQ itself don't use ValueSources at all (I never use them because I
> never understood how they are intended to work). Providing an own
> CustomScoreProvider is in most cases the easiest you can do. Just follow my
> advice from above to get the FieldCache in the ctor of the provider (from
> the reader). Don't call super in the provider just return your own score
> (some formula using subQueryScore and FieldCache, in most cases a simple
> multiplication).
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Dominika Puzio [mailto:dominika.puzio@gmail.com]
>> Sent: Tuesday, February 21, 2012 10:27 AM
>> To: java-user@lucene.apache.org
>> Subject: Re: Question about CustomScoreQuery
>>
>> Thanks for your answer. I checked what explain() says about my queries,
> and:
>>
>> MatchAllDocsQuery:
>> 1.0 = (MATCH) MatchAllDocsQuery, product of:
>>     1.0 = queryNorm
>>
>> FieldScoreQuery:
>> 0.5 = (MATCH) float(ratio), product of:
>>     0.5 = float(ratio)=0.5
>>     1.0 = boost
>>     1.0 = queryNorm
>>
>> CustomScoreQuery:
>> 0.24999999 = (MATCH) custom(*:*, float(ratio)), product of:
>>     0.24999999 = custom score: product of:
>>       0.70710677 = (MATCH) MatchAllDocsQuery, product of:
>>         0.70710677 = queryNorm
>>       0.35355338 = (MATCH) float(ratio), product of:
>>         0.5 = float(ratio)=0.5
>>         1.0 = boost
>>         0.70710677 = queryNorm
>>     1.0 = queryBoost
>>
>> so the reason is queryNorm.
>> I've done this test, because I wanted to write my own CustomScoreQuery and
> I
>> was trying to understand what's happening in customScore(int doc, float
>> subQueryScore, float valSrcScore) method of CustomScoreProvider. I
> assumed,
>> that if I use FieldScoreQuery as valueSourceQuery, valSrcScore parameter
> will
>> be equal to field value,  but it's not true. So now I'm using FieldCache
> to obtain
>> field value from index and use it in my own score function (there is one
> value
>> per document in this field in my index). It looks something like this:
>>
>> public float customScore(int doc, float subQueryScore, float valSrcScore)
>>              throws IOException {
>>
>>      float fieldValue = FieldCache.DEFAULT.getFloats(reader,
> "ratio")[doc];
>>
>>      float newValSrcScore = someFunction(someQueryTimeVariable,
>> fieldValue);
>>
>>      return super.customScore(doc, subQueryScore, newValSrcScore); }
>>
>> Am I right, or is there a better way to get field value?
>>
>> --
>> Dominika
>>
>>
>> On 20.02.2012 18:38, Ian Lea wrote:
>>> I can't explain this.  Can you get at an oal.search.Explanation?  You
>>> could write your own CustomScoreProvider - that might help you to
>>> double check what is being passed to it, and/or allow you to provide
>>> your own calculation.
>>>
>>>
>>> --
>>> Ian.
>>>
>>>
>>> On Thu, Feb 16, 2012 at 8:32 PM, Dominika Puzio
>>> <dominika.puzio@gmail.com>   wrote:
>>>> Hello,
>>>> I'm trying to understand the behavior of CustomScoreQuery. It seemed to
>> me,
>>>> that default
>>>> CustomScoreQuery(Query subQuery, ValueSourceQuery valSrcQuery)
>>>> should return a score that is a product of subQuery score
>>>> and valSrcQuery score. So I wrote a simple test case given below:
>>>>
>>>>        @Test
>>>>        public void createAndQueryIndex() throws CorruptIndexException,
>>>> LockObtainFailedException, IOException {
>>>>
>>>>                RAMDirectory dir = new RAMDirectory();
>>>>                IndexWriter writer = new IndexWriter(dir,
>>>>                                new IndexWriterConfig(Version.LUCENE_35,
>>>> new StandardAnalyzer(
>>>>                                                Version.LUCENE_35)));
>>>>
>>>>                Document doc = new Document();
>>>>                doc.add(new Field("name", "name", Store.YES,
>> Index.NOT_ANALYZED));
>>>>                doc.add(new Field("ratio", "0.5", Store.YES,
> Index.NOT_ANALYZED));
>>>>
>>>>                writer.addDocument(doc);
>>>>                writer.close();
>>>>
>>>>
>>>>                IndexSearcher searcher = new
>> IndexSearcher(IndexReader.open(dir));
>>>>
>>>>                Query matchAllDocs = new MatchAllDocsQuery();
>>>>                float score1 = searcher.search(matchAllDocs,
>>>> 1).scoreDocs[0].score;
>>>>                Assert.assertEquals(1.0f, score1, 0);
>>>>
>>>>                ValueSourceQuery fieldScoreQuery = new
> FieldScoreQuery("ratio",
>>>> FieldScoreQuery.Type.FLOAT);
>>>>                float score2 = searcher.search(fieldScoreQuery,
>>>> 1).scoreDocs[0].score;
>>>>                Assert.assertEquals(0.5f, score2, 0);
>>>>
>>>>
>>>>                CustomScoreQuery csq = new CustomScoreQuery(matchAllDocs,
>>>> fieldScoreQuery);
>>>>                float score3 = searcher.search(csq,
> 1).scoreDocs[0].score;
>>>>                Assert.assertEquals(score1*score2, score3, 0);
>>>>
>>>>                searcher.close();
>>>>        }
>>>>
>>>> But this test fails in third assertion - score3 is 0.249. Can someone
>>>> explain why? How the score3 is computed in this case?
>>>>
>>>> --
>>>> Regards
>>>> Dominika Puzio
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--
Dominika Puzio
Zespół Usług Wyszukiwania, SEO i Ergonomii
pok. 133(GDA), tel. 52.15.325


"WIRTUALNA POLSKA" Spolka Akcyjna z siedziba w Gdansku przy ul. Traugutta 115 C, wpisana do
Krajowego Rejestru Sadowego - Rejestru Przedsiebiorcow prowadzonego przez Sad Rejonowy Gdansk
- Polnoc w Gdansku pod numerem KRS 0000068548, o kapitale zakladowym 67.980.024,00 zlotych
oplaconym w calosci oraz Numerze Identyfikacji Podatkowej 957-07-51-216.

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message