Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (nike.apache.org: domain of peterlkeegan@gmail.com
 designates 209.85.198.229 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type;
        b=UWT5MQGjSqZPmzY0SzokLyaeJneG/zJp86FqROln2A+C98Z5mUswmeCgVgt2JNhhEP
         rTi7/CMv9YXdXZ4Y3yLcnM9m7NkHtX5BoBwgxQPeD0LoOxxFSyU/cIsXsRUp8X3PKyp/
         GALp2Mk7nAbPNEOAFtETiasW2hKvzHodpiLuM=
MIME-Version: 1.0
Date: Fri, 20 Feb 2009 16:03:00 -0500
Message-ID: <e994873a0902201303y62d320aoa7d92a89fc7fac7f@mail.gmail.com>
Subject: queryNorm affect on score
From: Peter Keegan <peterlkeegan@gmail.com>
To: java-user <java-user@lucene.apache.org>
Content-Type: multipart/alternative; boundary=000e0cd2e25869da1804635ffacc

--000e0cd2e25869da1804635ffacc
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

The explanation of scores from the same document returned from 2 similar
queries differ in an unexpected way. There are 2 fields involved, 'contents'
and 'literals'. The 'literals' field has setBoost = 0. As you an see from
the explanations below, the total weight of the matching terms from the
'literal' field is 0. However, the weights produced by the matching terms in
the 'contents' field is very different, even with the same matching terms.
The reason is that the 'queryNorm' value is very different because the
'sumOfSquaredWeights' is very different. Why is this?

First query: +(+contents:sales +contents:representative) +literals:jb$1
Explanation:
32.274593  sum of:
  32.274593  sum of:
    10.336284  weight(contents:sales in 14578), product of:
      0.54963183  queryWeight(contents:sales), product of:
        2.6595461  idf(contents: sales=83179)
        0.20666377  queryNorm
      18.805832  fieldWeight(contents:sales in 14578), product of:
        7.071068  btq, product of:
          1.4142135  tf(phraseFreq=3.0)
          5.0  scorePayload(...)
        2.6595461  idf(contents: sales=83179)
        1.0  fieldNorm(field=contents, doc=14578)
    21.93831  weight(contents:representative in 14578), product of:
      0.8007395  queryWeight(contents:representative), product of:
        3.8746004  idf(contents: representative=24678)
        0.20666377  queryNorm
      27.397562  fieldWeight(contents:representative in 14578), product of:
        7.071068  btq, product of:
          1.4142135  tf(phraseFreq=2.0)
          5.0  scorePayload(...)
        3.8746004  idf(contents: representative=24678)
        1.0  fieldNorm(field=contents, doc=14578)
  0.0  weight(literals:jb$1 in 14578), product of:
    0.23816177  queryWeight(literals:jb$1), product of:
      1.1524118  idf(docFreq=375455, numDocs=436917)
      0.20666377  queryNorm
    0.0  fieldWeight(literals:jb$1 in 14578), product of:
      1.0  tf(termFreq(literals:jb$1)=1)
      1.1524118  idf(docFreq=375455, numDocs=436917)
      0.0  fieldNorm(field=literals, doc=14578)


Second query: +(+contents:sales +contents:representative) +(literals:jb$1
literals:jb$9999)
Explanation:
10.550879  sum of:
  10.550879  sum of:
    3.3790317  weight(contents:sales in 14578), product of:
      0.17967999  queryWeight(contents:sales), product of:
        2.6595461  idf(contents: sales=83179)
        0.0675604  queryNorm
      18.805832  fieldWeight(contents:sales in 14578), product of:
        7.071068  btq, product of:
          1.4142135  tf(phraseFreq=3.0)
          5.0  scorePayload(...)
        2.6595461  idf(contents: sales=83179)
        1.0  fieldNorm(field=contents, doc=14578)
    7.171847  weight(contents:representative in 14578), product of:
      0.26176953  queryWeight(contents:representative), product of:
        3.8746004  idf(contents: representative=24678)
        0.0675604  queryNorm
      27.397562  fieldWeight(contents:representative in 14578), product of:
        7.071068  btq, product of:
          1.4142135  tf(phraseFreq=2.0)
          5.0  scorePayload(...)
        3.8746004  idf(contents: representative=24678)
        1.0  fieldNorm(field=contents, doc=14578)
  0.0  product of:
    0.0  sum of:
      0.0  weight(literals:jb$1 in 14578), product of:
        0.0778574  queryWeight(literals:jb$1), product of:
          1.1524118  idf(docFreq=375455, numDocs=436917)
          0.0675604  queryNorm
        0.0  fieldWeight(literals:jb$1 in 14578), product of:
          1.0  tf(termFreq(literals:jb$1)=1)
          1.1524118  idf(docFreq=375455, numDocs=436917)
          0.0  fieldNorm(field=literals, doc=14578)
    0.5  coord(1/2)


Peter

--000e0cd2e25869da1804635ffacc--