lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Toke Eskildsen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-2735) First Cut at GroupVarInt with FixedIntBlockIndexInput / Output
Date Tue, 02 Nov 2010 21:52:31 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12927635#action_12927635
] 

Toke Eskildsen commented on LUCENE-2735:
----------------------------------------

I tried making an extra test in GVintMicroBenchmark that created the same output as groupintsTest
but just read all the bytes directly back, no decoding, using IndexOutput. It's called GroupVarIntRead
in the output below and as can be seen, most of the processing seems to take place outside
of GVint decoding. Sorry no patch, as I messed up the formatting.

{code}
Running 4 Million random ints with max value: 64
GroupVarInt time per value: 4ns - time to decode 4M ints: 18ms
GroupVarIntRead time per value: 3ns - time to decode 4M ints: 15ms
Vint time per value: 6ns - time to decode 4M ints: 25ms

Running 4 Million random ints with max value: 1024
GroupVarInt time per value: 9ns - time to decode 4M ints: 39ms
GroupVarIntRead time per value: 5ns - time to decode 4M ints: 23ms
Vint time per value: 11ns - time to decode 4M ints: 44ms

Running 4 Million random ints with max value: 524288
GroupVarInt time per value: 12ns - time to decode 4M ints: 51ms
GroupVarIntRead time per value: 9ns - time to decode 4M ints: 38ms
Vint time per value: 14ns - time to decode 4M ints: 56ms

Running 4 Million random ints with max value: 67108864
GroupVarInt time per value: 14ns - time to decode 4M ints: 59ms
GroupVarIntRead time per value: 11ns - time to decode 4M ints: 45ms
Vint time per value: 15ns - time to decode 4M ints: 61ms
{code}


> First Cut at GroupVarInt with FixedIntBlockIndexInput / Output
> --------------------------------------------------------------
>
>                 Key: LUCENE-2735
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2735
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Index
>    Affects Versions: 4.0
>            Reporter: Simon Willnauer
>            Priority: Minor
>             Fix For: 4.0
>
>         Attachments: LUCENE-2735.patch, LUCENE-2735.patch, LUCENE-2735_alt.patch
>
>
> I have hacked together a FixedIntBlockIndex impl with Group VarInt encoding - this does
way worse than standard codec in benchmarks but I guess that is mainly due to the FixedIntBlockIndex
limitations. Once LUCENE-2723 is in / or builds with trunk again I will update and run some
tests. The isolated microbenchmark shows that there could be improvements over vint even in
java though and I am sure we can make it faster impl. wise.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message