hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mithun Radhakrishnan (JIRA)" <>
Subject [jira] [Created] (HIVE-12627) Hadoop23Shims.runDistCp() skips CRC checks.
Date Wed, 09 Dec 2015 01:19:11 GMT
Mithun Radhakrishnan created HIVE-12627:

             Summary: Hadoop23Shims.runDistCp() skips CRC checks.
                 Key: HIVE-12627
             Project: Hive
          Issue Type: Bug
            Reporter: Mithun Radhakrishnan

{{Hadoop23Shims.runDistCp()}} seems to be skipping CRC-checks. That setting opens the door
to bad data copy/commit. Is there a reason why we're doing this?

It's possible that if the final path is a file-system whose default block-sizes differ from
the source, the checksum-checks for the copy could fail. But since we're preserving the files'
block-sizes, this shouldn't be a concern.

Why are we skipping checksum checks? Can that be removed?

This message was sent by Atlassian JIRA

View raw message