hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6376) Distcp data between two HA clusters requires another configuration
Date Mon, 25 Aug 2014 21:39:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109787#comment-14109787
] 

Haohui Mai commented on HDFS-6376:
----------------------------------

bq. It looks like some folks have picked this up and moved it forward, is there a need for
me to do anything? From my earlier comments, there are some changes in dfsclusterhealth.jsp
in the Hadoop2 branch so that only NN's associated with "this" cluster are listed. Not sure
where that jsp file is for Hadoop 3.

It would be great if you can test the patch in your environment as well. The JSP UI has been
deprecated in hadoop 2 and removed from trunk.

> Distcp data between two HA clusters requires another configuration
> ------------------------------------------------------------------
>
>                 Key: HDFS-6376
>                 URL: https://issues.apache.org/jira/browse/HDFS-6376
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: datanode, federation, hdfs-client
>    Affects Versions: 2.2.0, 2.3.0, 2.4.0
>         Environment: Hadoop 2.3.0
>            Reporter: Dave Marion
>            Assignee: Dave Marion
>             Fix For: 3.0.0
>
>         Attachments: HDFS-6376-2.patch, HDFS-6376-3-branch-2.4.patch, HDFS-6376-4-branch-2.4.patch,
HDFS-6376-5-trunk.patch, HDFS-6376-6-trunk.patch, HDFS-6376-7-trunk.patch, HDFS-6376-branch-2.4.patch,
HDFS-6376-patch-1.patch, HDFS-6376.000.patch, HDFS-6376.008.patch, HDFS-6376.009.patch
>
>
> User has to create a third set of configuration files for distcp when transferring data
between two HA clusters.
> Consider the scenario in [1]. You cannot put all of the required properties in core-site.xml
and hdfs-site.xml for the client to resolve the location of both active namenodes. If you
do, then the datanodes from cluster A may join cluster B. I can not find a configuration option
that tells the datanodes to federate blocks for only one of the clusters in the configuration.
> [1] http://mail-archives.apache.org/mod_mbox/hadoop-user/201404.mbox/%3CBAY172-W2133964E0C283968C161DD1520%40phx.gbl%3E



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message