lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Muir (JIRA)" <>
Subject [jira] Commented: (LUCENE-1410) PFOR implementation
Date Tue, 21 Dec 2010 21:46:04 GMT


Robert Muir commented on LUCENE-1410:

I'm running into a nocommit for the nio byte buffer allocation in
Shall I try and move the buffer handling from there into FORIndexInput and PForDeltaIndexInput
at the codecs?

I am to blame for this I think! Actually I think the buffer handling could stay and we could
just remove the nocommit?
I've tested everything I can think of and it seems this nio ByteBuffer/IntBuffer approach
is always the fastest:
its only slower to do it other ways, and it doesnt help to do trickier things like IntBuffer
views of MMap even.

One thing that would be good, is it possible to encode the length in decompressed bytes (or
the length in bytes of exceptions)
into PFOR's int header? this would allow us to remove the wasted per-block int that we currently
encode now.

Then we could "put FOR and PFOR back together" again... sorry i split apart the decompressors
to remove the wasted int
in the FOR case since we can get it from its header already.

> PFOR implementation
> -------------------
>                 Key: LUCENE-1410
>                 URL:
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Index
>            Reporter: Paul Elschot
>            Priority: Minor
>             Fix For: Bulk Postings branch
>         Attachments: autogen.tgz, for-summary.txt, LUCENE-1410-codecs.tar.bz2, LUCENE-1410.patch,
LUCENE-1410.patch, LUCENE-1410.patch, LUCENE-1410.patch, LUCENE-1410b.patch, LUCENE-1410c.patch,
LUCENE-1410d.patch, LUCENE-1410e.patch, TermQueryTests.tgz,,,
>   Original Estimate: 21840h
>  Remaining Estimate: 21840h
> Implementation of Patched Frame of Reference.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message