Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (herse.apache.org: domain of deinspanjer@gmail.com
 designates 209.85.134.187 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=beta;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=Loac0qgqPHeQsbxko4b46iwfON1UHRE22pCBkfqlGgVDFU4Ob4NZ/VatWDF+b3c9pe7h8kgtTBI5dGhUxKmqq8xNnJuunsDz1cmAPCsH0mldK9/hmVC8rbbD7RIsU4ylKdI0Qw4DgD5jG31ZOWEAVbNeoMMT8102VslwMCyf7Sk=
Message-ID: <f81fa4180705050718lebb9119t964003b7a8f2be3f@mail.gmail.com>
Date: Sat, 5 May 2007 10:18:44 -0400
From: "Daniel Einspanjer" <deinspanjer@gmail.com>
To: java-user@lucene.apache.org, solr-user@lucene.apache.org
Subject: Re: Ideas for a relevance score that could be considered stable
 across multiple searches with the same query structure?
In-Reply-To: <Pine.LNX.4.58.0704111528010.5697@hal.rescomp.berkeley.edu>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_67009_19515380.1178374724925"
References: <f81fa4180704100504q142073c2n5ad0e88f0fff7579@mail.gmail.com>
	 <f81fa4180704101703x3c48d42ei529da26c6c8132ac@mail.gmail.com>
	 <0315F020-F9E4-41FB-AABA-E02D2B5D0649@apache.org>
	 <f81fa4180704110704mef44a69j1d1b90ead93f0e30@mail.gmail.com>
	 <Pine.LNX.4.58.0704111528010.5697@hal.rescomp.berkeley.edu>

------=_Part_67009_19515380.1178374724925
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

On 4/11/07, Chris Hostetter <hossman_lucene@fucit.org> wrote:
>
>
> A custom Similaity class with simplified tf, idf, and queryNorm functions
> might also help you get scores from the Explain method that are more
> easily manageable since you'll have predictible query structures hard
> coded into your application.
>
> ie: run the large query once, get the results back, and for each result
> look at the explanation and pull out the individual pieces of hte
> explanation and compare them with those of hte other matches to create
> your own "normalization".


Chuck Williams mentioned a proposal he had for normalization of scores that
would give a constant score range that would allow comparison of scores.
Chuck, did you ever write any code to that end or was it just algorithmic
discussion?

Here is the point I'm at now:

I have my matching engine working.  The fields to be indexed and the queries
are defined by the user.  Hoss, I'm not sure how that affects your idea of
having a custom Similarity class since you mentioned that having predictable
query structures was important...
The user kicks off an indexing then defines the queries they want to try
matching with.  Here is an example of the query fragments I'm working with
right now:
year_str:"${Year}"^2 year_str:[${Year -1} TO ${Year +1}]
title_title_mv:"${Title}"^10 title_title_mv:${Title}^2
+(title_title_mv:"${Title}"~^5 title_title_mv:${Title}~)
director_name_mv:"${Director}"~2^10 director_name_mv:${Director}^5
director_name_mv:${Director}~.7

For each item in the source feed, the variables are interpolated (the query
term is transformed into a grouped term if there are multiple values for a
variable). That query is then made to find the overall best match.
I then determine the relevance for each query fragment.  I haven't written
any plugins for Lucene yet, so my current method of determining the
relevance is by running each query fragment by itself then iterating through
the results looking to see if the overall best match is in this result set.
If it is, I record the rank and multiply that rank (e.g. 5 out of 10) by a
configured fragment weight.

Since the scores aren't normalized, I have no good way of determining a poor
overall match from a really high quality one. The overall item could be the
first item returned in each of the query fragments.

Any help here would be very appreciated. Ideally, I'm hoping that maybe
Chuck has a patch or plugin that I could use to normalize my scores such
that I could let the user do a matching run, look at the results and
determine what score threshold to set for subsequent runs.

Thanks,
Daniel

------=_Part_67009_19515380.1178374724925--