hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From selva <selvai...@gmail.com>
Subject High IO Usage in Datanodes due to Replication
Date Sat, 27 Apr 2013 18:03:21 GMT
Hi All,

I have lost amazon instances of my hadoop cluster. But i had all the data
in aws EBS volumes. So i launched new instances and attached volumes.

But all of the datanode logs keep on print the below lines it cauased to
high IO rate. Due to IO usage i am not able to run any jobs.

Can anyone help me to understand what it is doing? Thanks in advance.

2013-04-27 17:51:40,197 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
10.157.10.242:10013,
storageID=DS-407656544-10.28.217.27-10013-1353165843727, infoPort=15075,
ipcPort=10014) Starting thread to transfer block
blk_2440813767266473910_11564425 to 10.168.18.178:10013
2013-04-27 17:51:40,230 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
10.157.10.242:10013,
storageID=DS-407656544-10.28.217.27-10013-1353165843727, infoPort=15075,
ipcPort=10014):Transmitted block blk_2440813767266473910_11564425 to /
10.168.18.178:10013
2013-04-27 17:51:40,433 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
blk_2442656050740605335_10906493 src: /10.171.11.11:60744 dest: /
10.157.10.242:10013
2013-04-27 17:51:40,450 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Received block
blk_2442656050740605335_10906493 src: /10.171.11.11:60744 dest: /
10.157.10.242:10013 of size 25431

Thanks
Selva

Mime
View raw message