hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "风雨无阻" <232341...@qq.com>
Subject 回复: Block should be additionally replicated on 1 more rack(s)
Date Mon, 14 Jul 2014 09:17:51 GMT
HI ,
  I didn't try Hadoop rebalancer 。Because I remember rebalancer only considers disk load,
and won't consider that data blocks which rack 。
  I can try 。Thank you for your reply 。‍





------------------ 原始邮件 ------------------
发件人: "Yehia Elshater";<y.z.elshater@gmail.com>;
发送时间: 2014年7月14日(星期一) 下午4:52
收件人: "user"<user@hadoop.apache.org>; 

主题: Re: Block should be additionally replicated on 1 more rack(s)



Hi,

Did you try Hadoop rebalancer ?


http://hadoop.apache.org/docs/r1.0.4/hdfs_user_guide.html#Rebalancer
 





On 14 July 2014 04:10, 风雨无阻 <232341339@qq.com> wrote:
 HI all:


After the cluster configuration rack awareness,run " hadoop fsck / " 
 A lot of the following error occurred:
 Replica placement policy is violated for blk_-1267324897180563985_11130670. Block should
be additionally replicated on 1 more rack(s).


Online said "The reason is that three copies on the same rack" .
 The solution is now:
hadoop dfs -setrep 4  /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
sleep N
hadoop dfs -setrep 3 /user/hive/warehouse/tbl_add_av_errorlog_android/dt=2013-08-24/000004_0
 But the speed is very slow。‍


What is a better good way to make HDFS become healthy‍?‍
 


Thanks,‍

Ma Jian
Mime
View raw message