hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinayakumar B <vinayakuma...@huawei.com>
Subject RE: 【data migrate from hdfs0.20.* to hdfs-2.0.5(HA)】
Date Fri, 26 Jul 2013 10:39:38 GMT
Upgrade needs to be done in two phases.

1.       Upgrade the data to 2.0.5 (Non-HA)

a.       Just install the 2.0.5 Hadoop NonHA cluster by pointing to old Name dir and data
dirs and start the cluster with ./start-dfs.sh -upgrade

b.      Verify the upgraded data.

c.       Once the upgrade is fine, then finalize the upgrade by calling ./hdfs dfsadmin –finalizeUpgrade

d.      After finalizing stop the cluster

2.       Migrating cluster to HA

a.       Now install one more NameNode and zookeeper for HA and update the configurations
in all other machines for HA

b.      Call ./hdfs namenode –initializeSharedEdits to initialize the shared storage

c.       Start the previous NameNode first along with ZKFC and let it become active.

d.      Call ./hdfs namenode –bootstrapStandby from the new Namenode. This will copy the
data from previous namenode.

e.      After this start namenode and zkfc in new machine.

f.        Start all  datanodes.

Hope this solves your issue.

Vinayakumar B

From: Binglin Chang [mailto:decstery@gmail.com]
Sent: 26 July 2013 12:53
To: user@hadoop.apache.org
Subject: Re: 【data migrate from hdfs0.20.* to hdfs-2.0.5(HA)】

have you looked at distcp over hftp?

On Fri, Jul 26, 2013 at 2:28 PM, Bing Jiang <jiangbinglover@gmail.com<mailto:jiangbinglover@gmail.com>>


Have you tried to find out a way to make data transformation between two hdfs cluster, which
are the different version.

In our environment, one data in 0.20.*, and we want to migrate into 2.0.5(HA).

I try to use distcp, but it occurs to rpc version dismatch.
org.apache.hadoop.ipc.RemoteException: Server IPC version 8 cannot communicate with client
version 4

Could anyone give me an efficient way to make it?


Bing Jiang
weibo: http://weibo.com/jiangbinglover
BLOG: www.binospace.com<http://www.binospace.com>
BLOG: http://blog.sina.com.cn/jiangbinglover
Focus on distributed computing, HDFS/HBase

View raw message