hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9613) Don't compare file checksums after copy if block size and checksum opt are not preserved for replicated files
Date Fri, 08 Jan 2016 06:39:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kai Zheng updated HDFS-9613:
----------------------------
    Description: While working on related issue, it was noticed there are some places in {{distcp}}
that's better to be improved and cleaned up. Particularly, after a file is coped to target
cluster, it will check the copied file is fine or not. For replicated files, when checking,
if the source block size and checksum option are not preserved while copying, we can avoid
comparing the file checksums, which may save some time.  (was: While working on related issue,
it was noticed there are some places in {{distcp}} that's better to be improved and cleaned
up. Particularly, after a file is coped to target cluster, it will check the copied file is
fine or not. When checking, better to check block size first, then the checksum, because the
later is a little expensive.)

> Don't compare file checksums after copy if block size and checksum opt are not preserved
for replicated files
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9613
>                 URL: https://issues.apache.org/jira/browse/HDFS-9613
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Kai Zheng
>            Assignee: Kai Zheng
>            Priority: Minor
>         Attachments: HDFS-9613-v1.patch, HDFS-9613-v2.patch
>
>
> While working on related issue, it was noticed there are some places in {{distcp}} that's
better to be improved and cleaned up. Particularly, after a file is coped to target cluster,
it will check the copied file is fine or not. For replicated files, when checking, if the
source block size and checksum option are not preserved while copying, we can avoid comparing
the file checksums, which may save some time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message