Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <1536181652.1223836124213.JavaMail.jira@brutus>
Date: Sun, 12 Oct 2008 11:28:44 -0700 (PDT)
From: "Paul Elschot (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Issue Comment Edited: (LUCENE-1410) PFOR implementation
In-Reply-To: <521761508.1222889624375.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12637272#action_12637272 ] 

paul.elschot@xs4all.nl edited comment on LUCENE-1410 at 10/12/08 11:27 AM:
-----------------------------------------------------------------

bq. Or... is this because a full block (header + bits + exception data)  may not be 0 mod 4? (Though we too could pad if necessary)

The exceptions can be 1, 2 or 4 bytes and there is one more coding possible, we might reserve that for long.
I'd like to use some padding space to get the exceptions aligned to their natural border, 2 bytes exceptions at even byte position, and 4 bytes exceptions at  (... mod 4 == 0) natural int border. That way the rest of the padding space (if any) could be used to align to natural int border.

bq. I'd really love to get a prototype integration working so that we can then do an end-to-end performance test (ie, actual searches). I'm still wondering how much speedups at this layer will actually affect overall search time.

The decoding speeds reported here so far are about the same as  the ones reported by Zhang in 2008. That means there is room for disk speeds to catch up with CPU speed. In other words, adding this for proximities (and maybe docids and freqs) should make lucene a natural fit for ssd's, see also ZhangfFig. 15.


      was (Author: paul.elschot@xs4all.nl):
    bq. Or... is this because a full block (header + bits + exception data)  may not be 0 mod 4? (Though we too could pad if necessary)

The exceptions can be 1, 2 or 4 bytes and there is one more coding possible, we might reserve that for long.
I'd like to use some padding space to get the exceptions aligned to their natural border, 2 bytes exceptions at even byte position, and 4 bytes exceptions at  (... mod 4 == 0) natural int border. That way the rest of the padding space (if any) could be used to align to natural int border.

bq. I'd really love to get a prototype integration working so that we can then do an end-to-end performance test (ie, actual searches). I'm still wondering how much speedups at this layer will actually affect overall search time.

I'm not entirely sure about this, but the (articificially high) decoding speeds reported here so far are almost 2000 (!) times higher than the ones reported in the articles for decoding speeds for actual query processing. That means there is quite a bit of room for disk speeds to catch up with CPU speed. In other words, adding this for proximities (and maybe docids and freqs) should make lucene a natural fit for ssd's.

  
> PFOR implementation
> -------------------
>
>                 Key: LUCENE-1410
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1410
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Other
>            Reporter: Paul Elschot
>            Priority: Minor
>         Attachments: autogen.tgz, LUCENE-1410b.patch, LUCENE-1410c.patch, TestPFor2.java, TestPFor2.java, TestPFor2.java
>
>   Original Estimate: 21840h
>  Remaining Estimate: 21840h
>
> Implementation of Patched Frame of Reference.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org