hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xun REN (JIRA)" <j...@apache.org>
Subject ***UNCHECKED*** [jira] [Commented] (HDFS-13916) Distcp SnapshotDiff not completely implemented for supporting WebHdfs
Date Wed, 19 Sep 2018 08:09:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620266#comment-16620266

Xun REN commented on HDFS-13916:

Meanwhile, I will add more unit tests in order to test cross file systems DistCP.

> Distcp SnapshotDiff not completely implemented for supporting WebHdfs
> ---------------------------------------------------------------------
>                 Key: HDFS-13916
>                 URL: https://issues.apache.org/jira/browse/HDFS-13916
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: distcp, webhdfs
>    Affects Versions: 3.0.1, 3.1.1
>            Reporter: Xun REN
>            Assignee: Xun REN
>            Priority: Major
>              Labels: easyfix, newbie, patch
>         Attachments: HDFS-13916.002.patch, HDFS-13916.patch
> [~ljain] has worked on the JIRA: https://issues.apache.org/jira/browse/HDFS-13052 to
provide the possibility to make DistCP of SnapshotDiff with WebHDFSFileSystem. However, in
the patch, there is no modification for the real java class which is used by launching the
command "hadoop distcp ..."
> You can check in the latest version here:
> [https://github.com/apache/hadoop/blob/branch-3.1.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java#L96-L100]
> In the method "preSyncCheck" of the class "DistCpSync", we still check if the file system
is DFS. 
> So I propose to change the class DistCpSync in order to take into consideration what
was committed by Lokesh Jain.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message