lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <>
Subject Re: BooleanWeight.normalize(float) doesn't normalize prohibited clauses?
Date Thu, 11 May 2006 22:12:27 GMT

: >    Boolean match = null;
: As for the thoughts question below: this java-dev, not c-dev :)

i could not for the life of me understand this comment untill i got to the
end of your message...

: null for false: long time no see...

...i'm not trying to use null for false, i'm using null to indicate that
wether or not a match occured has not been explicitly specified -- it can
only be infered from the "value" of the explanation.  "true" means a
definitive match, and "false" means a definitive non-match.

: As long as there is no match, there will be no score, and no score could
: also be represented by NaN, so one might by default initialize the score
: value to NaN, drop setMatch() and isMatch() above, and have only:
: public Boolean getMatch() { return ! Float.isNaN(score); }

I assume by "score" you mean "value" (Explanations don't have a score
attribute, just a value attribute).  I don't want to go down the road of
assuming a match based on some special value of of the value -- that's the
cause of the current problems.  NaN is admitedly a better choice then
"0.0", but it's still a value that could concievably come up when scoring
a document for some as yet non-existent Scorer.

what i really want is for the Explanation class to precisely model the
same information as a Scorer returns for each doc...

   1) if scorer.doc() would ever return X, then the Explanation for that X
      should have a boolean indicating a "match"
   2) whatever value is returned by scorer.score() when scorer.doc() is
      returning X should be what the Explanation for X returns when you
      call getValue().

: But I'm not yet sure wether that would work in all cases.
: Is it possible/thinkable for a (sub)query to have a score value for a
: document, but no match against the same document?

I'm not sure if it can ever exist, since you currently can't ask a Scorer
for the score of a document unless the document matches, but the converse
is certainly true: a document can match on a query but have a score of 0,
or less then 0, or NaN ... that's what i'm trying to deal with, i want to
be able to model all of those cases in an Explanation object.

: That would be avoided by having getMatch() only. Once setMatch is called,
: getMatch would return false, except when setMatch is given a NaN, but
: that is probably not done in the current Lucene code.

Right.  currently *most* Explanations get a 0.0 value set if it's a non
match (some of them don't work at all for non-matches) .. which is why if
an explicit match boolean isn't specified, I want the fall back assumption
to be based on wether the value is 0.0 -- because that's the current
method for determining a match, and it will work with legacy custom Query
types people may have written which aren't in the lucene code base.

: > 4) change BooleanWeight.explain to call isMatch on the sub-explanations
: > when testing prohibited/required clauses.
: Or call getMatch(), whichever is implemented. This makes explaining the

i want to impliment both ... getMatch() existing for people that want to
know the exact state of the match (and will check for null to determine if
hte exact state is unknown) and isMatch for people who want the "best
guess" behavior ... which is not encapsulated in a method, instead of it
just being "convention"

: It might even become practical to use the explain() methods of the scorers
: that BooleanScorer2 is using. Only ConjunctionScorer would need
: an implementation of explain() in that case.

you lost me there ... BooleanScorer2 doesn't impliment explain (it throws
UnsupportedOperationException).  As far as i can tell almost none of the
Scorers in lucene core have good explain(int) implimentations.  most throw
UnsupportedOperationException, and the ones that don't tend to not be
correct (DisjunctionSumScorer for example never sets the value, so under
the current semantics it's explanations allways indicate failure)

Fortunately, I've yet to see a single Weight that delegates to it's Scorer
for building an Explanation.

: > ...this would be backwards compatible for any non-core Query classes out
: > there, and (as far as i can figure) be no worse then the current behavior
: > of testing an explanation.getValue() == 0.0f  9since that's the fallback
: > inside of isMatch())
: With the implementation above, the current code would have to be
: changed for the case when a 0.0f score value is used to indicate no match
: in an explanation: in that case no call to setValue() should be done.

I'm not following you ... even if existing code doesn't call setValue, the
additions i'm suggestion would assume "no match" unless setMatch was
called -- but it would expose the fact that this was only an assumption
for anyone who wanted to know (which isn't currently possible)


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message