hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shady Xu <shad...@gmail.com>
Subject How to distcp data between two clusters which are not in the same local network?
Date Mon, 15 Aug 2016 08:06:39 GMT
Hi all,

Recently I tried to use distcp to copy data across two clusters which are
not in the same local network. Fortunately, the nodes of the source cluster
each has an extra interface and ip which can be accessed from the
destination cluster. But during the process of distcp, the map tasks always
used the local ip of the source cluster nodes which they cannot reach.

I tried changing the property 'dfs.datanode.dns.interface' to the one I
want, and I tried changing the property 'dfs.datanode.use.datanode.hostname'
to true too. Nothing works.

Does hadoop now support this or do I miss something?

Mime
View raw message