hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
Date Fri, 21 Oct 2011 18:08:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13132880#comment-13132880
] 

Doug Cutting commented on HADOOP-7761:
--------------------------------------

No, i don't think it's a blocker at all.  It'd be great to get rid of all 1036 warnings, and
the only way to do it seems to be to by segregating code that uses Sun proprietary APIs, which
might be a good idea anyway.  But that's a separate issue that should not block this issue.
                
> Improve performance of raw comparisons
> --------------------------------------
>
>                 Key: HADOOP-7761
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7761
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, performance, util
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7761.txt
>
>
> Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe
to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more
CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into
WritableComparator.compareBytes to improve CPU performance in the shuffle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message