lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Elschot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-5293) Also use EliasFanoDocIdSet in CachingWrapperFilter
Date Sun, 20 Oct 2013 19:26:44 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800192#comment-13800192
] 

Paul Elschot commented on LUCENE-5293:
--------------------------------------

About the patch of 20 Oct 2013:

There is an EliasFanoEncoder2 with a constructor with only an upperBound.
This class delegates to EliasFanoEncoder and has very much the same methods, but not yet a
common interface/superclass.
It works by growing by a factor of 2 as necessary, and reencoding completely.
This reencoding could be optimized for the lower bits by using PackedInts to block decode/encode,
but I have not taken the time to implement that.

The EliasFanoDocIdSet has an extra constructor that takes only an upperbound that uses EliasFanoEncoder2.

I ran some tests on this, it appears to work fine here.
A possibly misplaced assert in TestEliasFanoDocIdSet is corrected to make the test pass.

CachingWrapperFilter uses the extra EliasFanoDocIdSet constructor when the number of doc ids
in the set is not known,
and it reverts to FBS when this EliasFanoDocIdSet is not sufficiently smaller than an FBS.

For the rest see my comments on the patch of 18 Oct.

I also did a little bit of perfomance testing, the memory usage of the upperbound only constructor
is higher, as expected.
The advance/next performance is only slightly less, again as expected.
I could not measure build times, I expect it to just about double for the upperbound only
constructor.


> Also use EliasFanoDocIdSet in CachingWrapperFilter
> --------------------------------------------------
>
>                 Key: LUCENE-5293
>                 URL: https://issues.apache.org/jira/browse/LUCENE-5293
>             Project: Lucene - Core
>          Issue Type: Improvement
>          Components: core/search
>            Reporter: Paul Elschot
>            Priority: Minor
>         Attachments: LUCENE-5293.patch, LUCENE-5293.patch, LUCENE-5293.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.1#6144)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message