hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15273) distcp error message on checksum mismatch is misleading when checksum protocol itself is different
Date Wed, 28 Feb 2018 15:22:00 GMT
Steve Loughran created HADOOP-15273:
---------------------------------------

             Summary: distcp error message on checksum mismatch is misleading when checksum
protocol itself is different
                 Key: HADOOP-15273
                 URL: https://issues.apache.org/jira/browse/HADOOP-15273
             Project: Hadoop Common
          Issue Type: Bug
          Components: tools/distcp
    Affects Versions: 3.1.0
            Reporter: Steve Loughran


When using distcp without {{-skipCRC}} . If there's a checksum mismatch between src and dest
store types (e.g hdfs to s3), then the error message will talk about blocksize, even when
its the underlying checksum protocol itself which is the cause for failure

bq. Source and target differ in block-size. Use -pb to preserve block-sizes during copy. Alternatively,
skip checksum-checks altogether, using -skipCrc. (NOTE: By skipping checksums, one runs the
risk of masking data-corruption during file-transfer.)

IF the checksum types are fundamentally different, the error message should say so



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message