Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: domain of clamprecht@gmail.com designates
 66.249.92.200 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references;
        b=EBeJpl3i+Y4frJMvqp0MwOkH9h49i0CPNV8IH3O8Tf2HflQpJf0d9g7/cL3cGW31O2pgSlCEkoPvG462oLcUmIzUitJRn0hbnZPXjuvcCkLA16FvpCCMKEMZvlqFcY+FzN5hkvnUGYxW/HHenSSq5vp58rasqcUgORPlaWwrZYo=
Message-ID: <88c6a6720602100008x55a8f0fat39ad05412fc8ddb3@mail.gmail.com>
Date: Fri, 10 Feb 2006 02:08:33 -0600
From: Chris Lamprecht <clamprecht@gmail.com>
To: java-user@lucene.apache.org
Subject: Re: Help: tweaking search - reducing IDF skew and implementing score
 cutoff
In-Reply-To: <6c33f9950602092325m2b2d54e6vaaa0a8b625280f1@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
References: <6c33f9950602092325m2b2d54e6vaaa0a8b625280f1@mail.gmail.com>

> 2. If I choose to sort the results by date, then recent documents with
> very very low relevancy (say the words searched appears only in
> content, and not in title/bylines/summary fields that are boosted
> higher) are still shown relatively high in the list, and I wish to
> omit them in general. What is the best way to implement some sort of a
> relevancy filter (include only documents with an normalized score of
> 0.2 or more....)? Or is there a better way around it?

As Chris pointed out, there isn't always an easy way to do this.  Your
suggestion of filtering below normalized scores of 0.2 might work,
assuming the most relevant document is 1.0.  You'll have to tune this
cutoff point and see how well it works.  One thing to watch out for is
that if the raw (non-normalized) score is less than 1.0, it is not
"normalized", so your most relevant document can have a score of less
than 1.0.  This may or may not be what you want, just something to
consider.  Lucene's Hits.java is where the normalization happens.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org