hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7761) Improve performance of raw comparisons
Date Wed, 26 Oct 2011 22:07:33 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13136471#comment-13136471

Eli Collins commented on HADOOP-7761:

Like the new FastByteComparisons class, let's put a javadoc pointer to com.google.common.primtives.UnsignedBytes
in the javadoc.

Why use lessThanUnsigned instead of UnsignedLongs#compare and why does the epilogue here differ
from upstream guava?

Otherwise looks great.
> Improve performance of raw comparisons
> --------------------------------------
>                 Key: HADOOP-7761
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7761
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io, performance, util
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hadoop-7761.txt, hadoop-7761.txt
> Guava has a nice implementation of lexicographical byte-array comparison that uses sun.misc.Unsafe
to compare unsigned byte arrays long-at-a-time. Their benchmarks show it as being 2x more
CPU-efficient than the equivalent pure-Java implementation. We can easily integrate this into
WritableComparator.compareBytes to improve CPU performance in the shuffle.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message