hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
Date Fri, 21 Oct 2011 16:44:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132812#comment-13132812

Doug Cutting commented on HADOOP-7761:

We could add <showWarnings>false</showWarnings> to the maven-compiler-plugin configuration
in pom.xml for code that generates warnings that we cannot disable, placing such code in a
separate module for this purpose...
> Improve performance of raw comparisons
> --------------------------------------
>                 Key: HADOOP-7761
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7761
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, performance, util
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7761.txt
> Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe
to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more
CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into
WritableComparator.compareBytes to improve CPU performance in the shuffle.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message