hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-7761) Improve performance of raw comparisons
Date Thu, 20 Oct 2011 17:34:10 GMT
Improve performance of raw comparisons

                 Key: HADOOP-7761
                 URL: https://issues.apache.org/jira/browse/HADOOP-7761
             Project: Hadoop Common
          Issue Type: Improvement
          Components: io, util
    Affects Versions: 0.23.0
            Reporter: Todd Lipcon
            Assignee: Todd Lipcon
         Attachments: hadoop-7761.txt

Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe
to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more
CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into
WritableComparator.compareBytes to improve CPU performance in the shuffle.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message