Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:MIME-Version:Content-Type:Message-ID;
  b=q/Ud7hHqElUCRhcFsNwjSYZClQEUZ5yTmkSYb5CXEOmHAFLD5iJfZflV0cLgcHRFzt1dYpSFQlzGimi67McvmNeoB/Y0JD4NzdzzTBDugLID8PLqB/gk3zpaNxc3I8ISL1XKfwrZzYgla3p1p/uXtTZYSINhCJi07aLURgNFTcI=;
Date: Mon, 19 Nov 2007 10:25:04 -0800 (PST)
From: HAIDUC SONIA <haiduc_sonia@yahoo.com>
Subject: Re: Scoring for all the documents in the index relative to a query
To: java-user@lucene.apache.org
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="0-1617004257-1195496704=:85955"
Message-ID: <870644.85955.qm@web60413.mail.yahoo.com>

--0-1617004257-1195496704=:85955
Content-Type: text/plain; charset=us-ascii

I am trying to order all the documents in the index according to their similarity to a given query. I am interested in having a complete list of *all* the documents in the index with their score. From what I understood by reading some documentation, Lucene internally assigns scores to all the documents in the index according to their similarity to the query, but when returning the hits, all the scores that are less than 0 are rounded to 0 and only the documents with the score > 0 are returned as hits. But what I would like to get is the list before this intermediate processing, so the list of all the documents with their raw score. I am trying to compare Lucene with LSI and for the comparison I want to do, I need the entire list of documents. Is there a way that I can get that with Lucene?
I hope I explained it clearly this time. If you need more details let me know.

Thank you,
Sonia

----- Original Message ----
From: Erick Erickson <erickerickson@gmail.com>
To: java-user@lucene.apache.org
Sent: Monday, November 19, 2007 11:55:00 AM
Subject: Re: Scoring for all the documents in the index relative to a query


Could you explain a bit more what problem you're trying to solve?
The reason I ask is that your question doesn't make sense to me,
since I have no idea what you expect by the term "negative score".

My simplistic view has been that all the docs returned via Hits
or HitCollector have scores > 0, and all the rest have scores of 0,
and this view is supported by the explanation of
HitCollector.collect

" Called once for every non-zero scoring document, with the
document number and its score."

You might also get value from this page:
http://lucene.apache.org/java/docs/scoring.html#Scoring

Best
Erick

On Nov 19, 2007 11:05 AM, HAIDUC SONIA <haiduc_sonia@yahoo.com> wrote:

> Hi everyone,
>
> I am trying to obtain the score for each document in the index
 relative to
> a given query. For example, if I have the query "search file", I am
 trying
> to get the list of all documents in the index and their scores
 relative to
> the given query. I tried first using Hits, which gave me the
 normalized
> score. I thought that I don't see the whole list of documents and
 their
> scores because of the normalization, so I tried using HitsCollector.
 But
> even after using HitsCollector, I get the same number of matching
 documents,
> so the normalization didn't exclude documents because of negative
 scoring.
> Does Lucene actually compute the score for all the documents in the
 index or
> just for matching documents? I really need to have the scores for all
 the
> documents in the index relative to the query (even if negative), not
 just
> the ones that contain the query terms(this is what Lucene considers
> "matching documents", right?). Is this possible using Lucene?
>
> I really appreciate your time and effort!
> Thanks,
> Sonia
>
>
>
>
>
>
  ____________________________________________________________________________________
> Get easy, one-click access to your favorites.
> Make Yahoo! your homepage.
> http://www.yahoo.com/r/hs
>


      ____________________________________________________________________________________
Get easy, one-click access to your favorites. 
Make Yahoo! your homepage.
http://www.yahoo.com/r/hs 
--0-1617004257-1195496704=:85955--