hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2130) Switch default checksum to CRC32C
Date Wed, 02 Nov 2011 20:15:32 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142476#comment-13142476

Eli Collins commented on HDFS-2130:

After paging in the relevant info I think approach #2 makes sense. I don't think we'll need
to change the chunk size and the slight performance hit is only when filling up the last block
of files being appended that were created in previous releases (they can always switch the
default if they really care, but they shouldn't).

+1 to the latest patch

> Switch default checksum to CRC32C
> ---------------------------------
>                 Key: HDFS-2130
>                 URL: https://issues.apache.org/jira/browse/HDFS-2130
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs client
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-2130.txt, hdfs-2130.txt
> Once the other subtasks/parts of HDFS-2080 are complete, CRC32C will be a much more efficient
checksum algorithm than CRC32. Hence we should change the default checksum to CRC32C.
> However, in order to continue to support append against blocks created with the old checksum,
we will need to implement some kind of handshaking in the write pipeline.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message