hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From cho ju il <tjst...@kgrid.co.kr>
Subject Balancing is very slow.
Date Thu, 11 Sep 2014 04:17:45 GMT
hadoop 2.4.1
Balancing is very slow. 
 
$HADOOP_PREFIX/bin/hdfs dfsadmin -setBalancerBandwidth 52428800
 
It takes long time to move the one block.
 
2014. 09. 11. 11:38:01  Block begins to move
2014-09-11  11:47:20  Complete block move
 
 
#10.2.1.211 netstat, Block begins to move, 10.2.1.210 --&gt;&gt;&gt; 10.2.1.211
2014. 09. 11. 11:38:01
tcp   1110650      0 10.2.1.211:56819            10.2.1.210:40010            ESTABLISHED -
 
 
# datanode log, 10.2.1.211 
2014-09-11 11:47:09,819 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Copied BP-1770955034-0.0.0.0-1401163460236:blk_1077753386_4013196
to /10.2.1.211:56819
 
# namenode balancer log
2014-09-11 11:47:20,782 INFO org.apache.hadoop.hdfs.server.balancer.Balancer: Successfully
moved blk_1077753386_4013196 with size=134217728 from 10.2.1.204:40010 to 10.2.1.211:40010
through 10.2.1.210:40010

# check network state, File transfer speed using scp,  76.7MB/sdummy.tar   100%  230MB  76.7MB/s
  00:03  
Mime
View raw message