hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6560) Byte array native checksumming on DN side
Date Tue, 24 Jun 2014 18:38:24 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14042491#comment-14042491
] 

Tsz Wo Nicholas Sze commented on HDFS-6560:
-------------------------------------------

By the Java implementation, do you mean PureJavaCrc32?  Which JDK are you using?

For Java 6, my patch in HADOOP-10674 will improve 40% - 90% performance.  For Java 7 or 8,
it turns out that the performance of java.util.zip.CRC32 is very good.  So, it becomes tricky
to select the best implementation across platforms.  We should add a performance test similar
to TestPureJavaCrc32.PerformanceTest for testing the native crc implementations.

> Byte array native checksumming on DN side
> -----------------------------------------
>
>                 Key: HDFS-6560
>                 URL: https://issues.apache.org/jira/browse/HDFS-6560
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode, hdfs-client, performance
>            Reporter: James Thomas
>            Assignee: Todd Lipcon
>         Attachments: HDFS-3528.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message