hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xun REN (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13916) Distcp SnapshotDiff not completely implemented for supporting WebHdfs
Date Wed, 26 Sep 2018 21:16:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16629422#comment-16629422

Xun REN commented on HDFS-13916:

Hi [~jojochuang] 

Could you or another committer can help to review the code and then close the Jira if it is
alright for you ?


> Distcp SnapshotDiff not completely implemented for supporting WebHdfs
> ---------------------------------------------------------------------
>                 Key: HDFS-13916
>                 URL: https://issues.apache.org/jira/browse/HDFS-13916
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: distcp, webhdfs
>    Affects Versions: 3.0.1, 3.1.1
>            Reporter: Xun REN
>            Assignee: Xun REN
>            Priority: Major
>              Labels: easyfix, newbie, patch
>         Attachments: HDFS-13916.002.patch, HDFS-13916.003.patch, HDFS-13916.004.patch,
HDFS-13916.005.patch, HDFS-13916.patch
> [~ljain] has worked on the JIRA: https://issues.apache.org/jira/browse/HDFS-13052 to
provide the possibility to make DistCP of SnapshotDiff with WebHDFSFileSystem. However, in
the patch, there is no modification for the real java class which is used by launching the
command "hadoop distcp ..."
> You can check in the latest version here:
> [https://github.com/apache/hadoop/blob/branch-3.1.1/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/DistCpSync.java#L96-L100]
> In the method "preSyncCheck" of the class "DistCpSync", we still check if the file system
is DFS. 
> So I propose to change the class DistCpSync in order to take into consideration what
was committed by Lokesh Jain.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message