hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dave Thompson (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-8233) Turn CRC checking off for 0 byte size and differing blocksizes
Date Fri, 30 Mar 2012 22:15:27 GMT
Turn CRC checking off for 0 byte size and differing blocksizes

                 Key: HADOOP-8233
                 URL: https://issues.apache.org/jira/browse/HADOOP-8233
             Project: Hadoop Common
          Issue Type: Bug
    Affects Versions: 0.23.3
            Reporter: Dave Thompson
            Assignee: Dave Thompson

DistcpV2 (hadoop-tools/hadoop-distcp/..) can fail from checksum failure, sometimes when copying
a 0 byte file.    Root cause of this may have to do with an inconsistent nature of HDFS when
creating 0 byte files, however distcp can avoid this issue by not checking CRC when size is

Further, distcp fails checksum when copying from two clusters that use different blocksizes.
 In this case it does not make sense to check CRC, as it is a guaranteed failure.

We need to turn CRC checking off for the above two cases.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message