lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Is it possible to normalise BM25 scores in the query level?
Date Tue, 27 Jun 2017 14:39:25 GMT
Hi,

Once you have executed the query the TopDocs collector gives you the maximum score. Then you
just need to normalize on your own.

But keep in mind: This is not always a good idea, because the maximum score and the score
of the first document does not mean to be useful to compare. E.g. if the first, top-ranking
result is a very bad match and no better ones are there, there is not reason to say it's a
100% hit!

BTW: Lucene up to version 6 had some internal normalization in place, but this was removed
in Lucene 7. The reason is simple: The scores are calculated just to compare them inside the
same result set. They were never implemented to be used across different indexes or queries.

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Rifat [mailto:rifatozcan1981@gmail.com]
> Sent: Tuesday, June 27, 2017 11:46 AM
> To: java-user@lucene.apache.org
> Subject: Is it possible to normalise BM25 scores in the query level?
> 
> Hi all,
> 
> I searched for this a lot but could not find a clear answer, yet. is there a
> way such that Lucene (or Elasticsearch) provides query level normalization
> of BM25 scores. Because BM25 scores varies considerably across queries. For
> example, is it possible to get scores normalised by the max score for that
> query? Since lucene processes docs one at a time and return score for that
> document, at that moment, it seems not easy to do the normalisation.
> 
> thanks,
> rifat
> 
> 
> 
> 
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Is-it-
> possible-to-normalise-BM25-scores-in-the-query-level-tp4342991.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message