lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Custom Query & reading plongs used by a custom Scorer
Date Fri, 06 Oct 2017 15:06:03 GMT
docValues are the first thing I'd look at. What you've done is an
anit-pattern for scoring because it reads the stored data from disk
and decompress it to read the value; as you say costly.

Getting it from a docValues field, OTOH, will read the value(s)
directly from MMapDirectory space, i.e. the OSs memory space. As an
aside, this is why Streaming only works with DV fields.

Two cautions though in terms of differences between DV and stored when
you have more than 1 term. The underlying structured is a sorted set,
therefore:
1> the contents are ordered by "natural" order rather than insertion order.
2> multiple identical values are collapsed into a single value.

So storing 1, 3, 99, 4, 4, 4, 4, 2, 3 will be returned a s 1, 2, 3, 4, 99

And another caution: you'll have to re-index completely when you add
docValues=true to your field definition, I'd start with a new
collection.

Best,
Erick

On Fri, Oct 6, 2017 at 7:32 AM, Dominik Safaric
<dominiksafaric@gmail.com> wrote:
> I've implemented a custom Query whose responsibilities are as follows.
> First, using an instance of a PointValues.IntersectVisitor classifying
> documents as hit or not using a plong value. Secondly, calculating custom
> scores using another document field, specified in the mapping as plongs.
> The later is expected to calculate the custom score using an array of longs
> comprised of 46 values.
>
> The problem I am having is performance wise. Namely for calculating the
> custom score I'm retrieving the values of the field using
> LeafReader.document(docId()) which is a costly process. What alternatives
> are there for reading plongs using a LeafReader and DocIdSetIterator within
> a custom Scorer implementation?
>
> Thanks in advance.
> Dominik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message