hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10778) Use NativeCrc32 only if it is faster
Date Thu, 03 Jul 2014 08:45:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051197#comment-14051197
] 

Steve Loughran commented on HADOOP-10778:
-----------------------------------------

This is interesting - the speedup may depend on the CPU as well as other factors. 

Maybe the number "512" could be provided by the NativeCRC code itself, so it can make those
decisions based on its knowledge of things ... the pending Arm patch could supply a different
number than x86 parts, etc.

> Use NativeCrc32 only if it is faster
> ------------------------------------
>
>                 Key: HADOOP-10778
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10778
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: util
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: c10778_20140702.patch
>
>
> From the benchmark post in [this comment|https://issues.apache.org/jira/browse/HDFS-6560?focusedCommentId=14044060&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14044060],
NativeCrc32 is slower than java.util.zip.CRC32 for Java 7 and above when bytesPerChecksum
> 512.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message