Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Date: Wed, 9 Dec 2015 01:19:11 +0000 (UTC)
From: "Mithun Radhakrishnan (JIRA)" <jira@apache.org>
To: dev@hive.apache.org
Message-ID: <JIRA.12920261.1449623929000.307844.1449623951101@Atlassian.JIRA>
In-Reply-To: <JIRA.12920261.1449623929000@Atlassian.JIRA>
References: <JIRA.12920261.1449623929000@Atlassian.JIRA>
 <JIRA.12920261.1449623929461@arcas>
Subject: [jira] [Created] (HIVE-12627) Hadoop23Shims.runDistCp() skips CRC
 checks.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Mithun Radhakrishnan created HIVE-12627:
-------------------------------------------

             Summary: Hadoop23Shims.runDistCp() skips CRC checks.
                 Key: HIVE-12627
                 URL: https://issues.apache.org/jira/browse/HIVE-12627
             Project: Hive
          Issue Type: Bug
            Reporter: Mithun Radhakrishnan


{{Hadoop23Shims.runDistCp()}} seems to be skipping CRC-checks. That setting opens the door to bad data copy/commit. Is there a reason why we're doing this?

It's possible that if the final path is a file-system whose default block-sizes differ from the source, the checksum-checks for the copy could fail. But since we're preserving the files' block-sizes, this shouldn't be a concern.

Why are we skipping checksum checks? Can that be removed?


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)