Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: pass (asf.osuosl.org: local policy)
From: Paul Elschot <paul.elschot@xs4all.nl>
To: java-user@lucene.apache.org
Subject: Re: Modifying the stored norm type
Date: Wed, 21 Jun 2006 19:40:41 +0200
User-Agent: KMail/1.8.2
References: <000901c69488$8705df90$9401a8c0@corp.keepmedia.int>
In-Reply-To: <000901c69488$8705df90$9401a8c0@corp.keepmedia.int>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200606211940.42044.paul.elschot@xs4all.nl>

On Tuesday 20 June 2006 18:42, Dan Climan wrote:
> >Paul Elschot <paul.elschot@xs4all.nl> 
> >>On Tuesday 20 June 2006 12:02, Marcus Falck wrote:
> >> After a lot of debugging and some API doc reading I have come to the
> > conclusion that the static encodeNorm method of the Similarity class
> > will encode my boost value into a single byte decimal number.
> >> 
> >> And I will loose a lot of resolution and will get severe rounding
> >> errors. 
> >> 
> >> Since I need the exact float value as boost representation this isn't
> >> good enough in my case.
> 
> >An exact float value is a bit of an oxymoron.
> >How exact do you need it to be? 
> 
> >The range of values that can be encoded by the existing encodeNorm()
> >and decodeNorm() is quite big (about 10e-10 to 10e+10 iirc),
> >and since there are only 255 possible values in there (excluding 0),
> >the rounding errors can be severe indeed.
> >However, with a smaller range, the rounding errors would also be smaller.
> 
> >Are 256 different values enough for your case?
> 
> >Regards,
> >Paul Elschot
> 
> I, too, have found that 256 values were not enough. I tried changing the
> encodeNorm function to use a narrower range of values, but the 256 limit
> makes it degrade quickly if I get any results outside the expected range.
> This was true when we tried various algorithms for boosting results based on
> external factors. 
> 
> FunctionQuery(not currently in core lucene) from the SOLR project may be be
> an alternative. Can it replace all uses of the norm?
> 
> Now that omitNorms is part of the core, the impact of allowing a 2 byte (or
> even 4 byte norm) is not nearly as severe on memory. Any suggestions for how
> to create a multi-byte norm as an option, but enable the same code to
> reading existing 1 byte norms?

No more than to add your needs here: 
http://wiki.apache.org/jakarta-lucene/FlexibleIndexing,
and to subscribe to java-dev.

Regards,
Paul Elschot

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org