lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Very high fieldNorm for a field resulting in bad results
Date Tue, 26 Sep 2006 19:06:57 GMT

: The symptom:
: Very high fieldNorm for field A.(explain output pasted below) The boost i am
: applying to the troublesome field is 3.5  & the max boost applied per doc is
: 1.8
: Given that information, the very high fieldNorm is very surprising to me.
: Based on what I read,  FieldNorm  = 1 / sqrt(sum of terms) , possibly
: multiplied by field boost values.

The value of a the field norm for any field named "A" is typically the
lengthNorm of the field, times the document boost, times the field boost
for *each* Field instance added to the document with the name "A".
(lengthNorm is by default 1/swrt(num of terms))

so in your situation...

: for (Collection of values){
:      Field thisField = new Field(fieldName, value, fieldConfig.STORED,
: fieldConfig.INDEXED);
:      thisField.setBoost(fieldConfig);
:     doc.add(thisField);

the fieldNorm for "A" is going to be fieldConfig * values.size() * any
document boost you didn't mention using * length norm.

: which should basically lead to the "values" being appended,
: Am i making a mistake in the way I am adding fields ?

the way you are adding fields is the "proper" way to deal with multi-value
fields in my opinion, but it may be leading to more "boost" then you
intended, in which case only boosting the first Field may be the way to
go.

another aspect of this to keep in mind, is htat since fieldNorms are
stored as a single byte encoded float, some precission is lost ... the
byte encoding for the norms is targeted at smaller values, so with really
big norms you might find the problem exaserbated by the rounding.

play around with your boost values - you can use indexReader.norms("A")
along with similarity.decodeNorm to see what norm values your various
documents are getting as you tweak your numbers.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message