hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9640) Remove hsftp from DistCp
Date Tue, 12 Jan 2016 17:32:40 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15094352#comment-15094352

Wei-Chiu Chuang commented on HDFS-9640:

Seems to me that if hftp/hsftp is removed, DistCp should use webhdfs/swebhdfs instead.

To use swebhdfs with DistCp, the curent implementation must have ssl-client.xml in class path
(such as /etc/hadoop/conf)  http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cdh_admin_distcp_data_cluster_migrate.html
Without it, distcp throws an SSLHandshakeException, and adding -mapredSslConf does not help
because the exception is thrown before checking -mapredSslConf.

I think we can do either
(1) keep the current behavior, remove -mapredSslConf 
(2) repurpose -mapredSslConf and use it to specify ssl-client.xml for SSL over WebHDFS (swebhdfs)

Any suggestions?

> Remove hsftp from DistCp
> ------------------------
>                 Key: HDFS-9640
>                 URL: https://issues.apache.org/jira/browse/HDFS-9640
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: distcp
>    Affects Versions: 3.0.0
>            Reporter: Wei-Chiu Chuang
>            Assignee: Wei-Chiu Chuang
> Per discussion in HDFS-9638,
> after HDFS-5570, hftp/hsftp are removed from Hadoop 3.0.0. But DistCp still makes reference
to hsftp via parameter -mapredSslConf. This parameter would be useless after Hadoop 3.0.0;
 therefore it should be removed, and then document the changes.
> This JIRA is intended to track the status of the code/docs change involving the removal
of hsftp in DistCp.

This message was sent by Atlassian JIRA

View raw message