hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
Date Mon, 06 Feb 2012 21:59:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13201649#comment-13201649

Thomas Graves commented on HADOOP-7761:

I think this should be merged into branch 0.23.
> Improve performance of raw comparisons
> --------------------------------------
>                 Key: HADOOP-7761
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7761
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, performance, util
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.24.0
>         Attachments: hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt, hadoop-7761.txt
> Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe
to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more
CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into
WritableComparator.compareBytes to improve CPU performance in the shuffle.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message