hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3528) Use native CRC32 in DFS write path
Date Tue, 24 Jun 2014 00:52:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14041545#comment-14041545

Tsz Wo Nicholas Sze commented on HDFS-3528:

Is the native library faster than the Java implementation only for CRC32C but not CRC32?

> Use native CRC32 in DFS write path
> ----------------------------------
>                 Key: HDFS-3528
>                 URL: https://issues.apache.org/jira/browse/HDFS-3528
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, hdfs-client, performance
>    Affects Versions: 2.0.0-alpha
>            Reporter: Todd Lipcon
>            Assignee: James Thomas
> HDFS-2080 improved the CPU efficiency of the read path by using native SSE-enabled code
for CRC verification. Benchmarks of the write path show that it's often CPU bound by checksums
as well, so we should make the same improvement there.

This message was sent by Atlassian JIRA

View raw message