lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrien Grand (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (LUCENE-4098) Efficient bulk operations for packed integer arrays
Date Thu, 31 May 2012 17:58:23 GMT

     [ https://issues.apache.org/jira/browse/LUCENE-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Adrien Grand updated LUCENE-4098:
---------------------------------

    Attachment: LUCENE-4098.patch

Here is the patch for the proposed modifications. All {{Mutable}} implementations have a new
efficient {{fill}} method and Packed64SingleBlock* classes also have efficient bulk get and
set.

For example, the execution time of the following (unrealistic) microbenchmark is more than
twice better with the patch applied on my computer thanks to the use of {{PackedInts.copy}}
instead of naive copy (see {{GrowableWriter#ensureCapacity}}).

{code}
for (int k = 0; k < 50; ++k) {
    long start = System.nanoTime();
    GrowableWriter wrt = new GrowableWriter(1, 1 << 22, PackedInts.DEFAULT);
    for (int i = 0; i < 1 << 22; ++i) {
        wrt.set(i,i);
    }
    long end = System.nanoTime();
    System.out.println((end - start) / 1000000);
    long sum = 0;
    for (int i = 0; i < wrt.size(); ++i) {
        sum += wrt.get(i);
    }
    System.out.println(sum);
}
{code}

                
> Efficient bulk operations for packed integer arrays
> ---------------------------------------------------
>
>                 Key: LUCENE-4098
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4098
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/other
>            Reporter: Adrien Grand
>            Priority: Minor
>             Fix For: 4.1
>
>         Attachments: LUCENE-4098.patch
>
>
> There are some places in Lucene code that {iterate over,set} ranges of values of a packed
integer array. Because bit-packing implementations (Packed*) tend be slower than direct implementations,
this can take a lot of time.
> For example, under some scenarii, GrowableWriter can take most of its (averaged) {{set}}
time in resizing operations.
> However, some bit-packing schemes, such as the one that is used by {{Packed64SingleBlock*}},
allow to implement efficient bulk operations such as get/set/fill. Implementing these bulk
operations in {{PackedInts.{Reader,Mutable}}} and using them across other components instead
of their single-value counterpart could help improve performance.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message