Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (athena.apache.org: domain of gsiasf@gmail.com designates
 209.85.221.171 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=sender:content-type:mime-version:subject:from:in-reply-to:date
         :content-transfer-encoding:message-id:references:to:x-mailer;
        b=I5phnOEugm65B++SciZRYbKB9GYefNNnVFJRqpVtCDOOZPuLqO2DaiI7T0ojy3baLM
         6obSLmPxocrffXFiKXyr6oVuwFN8uF/ijdMC//bsoPakTUgoI8OfpH56mEL/OP0ZJqHb
         7+FkD91+MmWT67PIZhBTDRSEFtllgIn5IWeOs=
Sender: Grant Ingersoll <gsiasf@gmail.com>
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1078)
Subject: Re: Relevancy Practices
From: Grant Ingersoll <gsingers@apache.org>
In-Reply-To: <y2re994873a1005031108g8d8a0153u161aaf008a03a7ab@mail.gmail.com>
Date: Wed, 5 May 2010 07:10:08 -0700
Content-Transfer-Encoding: quoted-printable
Message-Id: <5CD2BE19-C279-41C9-A353-F25C5F5138AE@apache.org>
References: <E6C119F7-C966-4280-94DD-3575BB3CE3F4@apache.org>
 <y2re994873a1005031108g8d8a0153u161aaf008a03a7ab@mail.gmail.com>
To: java-user@lucene.apache.org

Thanks, Peter.

Can you share what kind of evaluations you did to determine that the end =
user believed the results were equally relevant?  How formal was that =
process?

-Grant

On May 3, 2010, at 11:08 AM, Peter Keegan wrote:

> We discovered very soon after going to production that Lucene's scores =
were
> often 'too precise'. For example, a page of 25 results may have =
several
> different score values, and all within 15% of each other, but to the =
end
> user all 25 results were equally relevant. Thus we wanted the =
secondary sort
> field to determine the order, instead. This required writing a custom =
score
> comparator to 'round' the scores. The same thing occurred for distance
> sorting. We also limit the effect of term frequency to help prevent
> spamming.  In comparison to Avi, we use 'AND' as the default operator =
for
> keyword queries and if no docs are found, the query is automatically =
retried
> with 'OR'. This improves precision a bit and only occurs if the user
> provides no operators.
>=20
> Lucene's Explanation class has been invaluable in helping me to =
explain a
> particular sort order in many, many situations.
> Most of our relevance tuning has occurred after deployment to =
production.
>=20
> Peter
>=20
> On Thu, Apr 29, 2010 at 10:14 AM, Grant Ingersoll =
<gsingers@apache.org>wrote:
>=20
>> I'm putting on a talk at Lucene Eurocon (
>> http://lucene-eurocon.org/sessions-track1-day2.html#1) on "Practical
>> Relevance" and I'm curious as to what people put in practice for =
testing and
>> improving relevance.  I have my own inclinations, but I don't want to =
muddy
>> the water just yet.  So, if you have a few moments, I'd love to hear
>> responses to the following questions.
>>=20
>> What worked?
>> What didn't work?
>> What didn't you understand about it?
>> What tools did you use?
>> What tools did you wish you had either for debugging relevance or =
"fixing"
>> it?
>> How much time did you spend on it?
>> How did you avoid over/under tuning?
>> What stage of development/testing/production did you decide to do =
relevance
>> tuning?  Was that timing planned or not?
>>=20
>>=20
>> Thanks,
>> Grant
>>=20


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org