hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shengjie Min <kelvin....@gmail.com>
Subject migration from hadoop cluster cdh3 to cdh4
Date Fri, 07 Dec 2012 13:27:53 GMT

Is there any instructions or documents covering migration from hadoop hdfs
cdh3 to cdh4 since all the docs I found are talking about in place
upgrading ONLY?

I have two hadoop clusters, My target is to use hadoop -cp to copy all the
hdfs files from *cluster1* to*cluster2*

*Cluster1:* Hadoop 0.20.2-cdh3u4

*Cluster2:* Hadoop 2.0.0-cdh4.1.1

Now, even just running dfs -ls command against *cluster1* remotely on *
cluster2* as below:

hadoop fs -ls hdfs://cluster1-namenode:8020/hbase

I am getting the exception:

*ls: Failed on local exception: java.io.IOException: Response is null.;
Host Details : local host is: "cluster2-namenode/10.21.xxx.xxx";
destination host is: "cluster1-namenode":8020;*
I think it's due to the hadoop version difference. In my case, cdh3 cluster
doesn't have mapred deployed which rules out all the distcp, bhase
copytable options. And the hbase replication ability is not available on
cdh3 cluster neither. I am struggling to think of a way to migrate the hdfs
data from *cluster1* to *cluster2.*

All the best,
Shengjie Min

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message