hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brahma Reddy Battula <brahmareddy.batt...@huawei.com>
Subject RE: namenode doesn't receive datanode deactivate event
Date Thu, 03 Jul 2014 04:32:17 GMT
A Heartbeat mechanism

Hadoop cluster is a master/slave mode, master includes namenode+resoucemanager, slave includes

Master starts, will open a IPC server there, waiting for the slave heartbeat.

Slave startup, will connect to the master, and every 3 seconds to master sends a heartbeat;,
this time through ” heartbeat.recheck.interval” property to set . To their own state information
told master, then master is through the heart of the return value, to the slave node to convey

You can tune heartbeat.recheck.interval  configuration..

The NameNode updates this detail after 10.5 minutes by default. You can see the dead and live
datanodes at that time.

It computes this heartbeatExpireInterval time by the following formula

  heartbeatExpireInterval = 2 * heartbeatRecheckInterval +
      10 * heartbeatInterval

where heartbeatRecheckInterval is defined by the configuration heartbeat.recheck.interval
which is 5 minutes by default and heartbeatInterval by dfs.heartbeat.interval which is 3 seconds
by default.

heartbeatExpireInterval = 10.5 minutes

Same for Yarn Also.

Thanks & Regards

Brahma Reddy Battula

From: MrAsanjar . [afsanjar@gmail.com]
Sent: Wednesday, July 02, 2014 11:12 PM
To: user@hadoop.apache.org
Subject: namenode doesn't receive datanode deactivate event

Hi all,
I have a small hadoop 2.2.0 development cluster consist of a master node ( namenode+resoucemanager
), and 4 slave nodes ( datanodes+nodemanager).
My configuration is as such that it enables me dynamically add slave nodes by executing commands:
.../sbin/hadoop-daemons.sh start datanode
.../sbin/yarn-daemons.sh start nodemanage
I could verify the activation of the new slave node by executing "jps" command (datanode and
 nodemanager are active ) on the newly created node and by monitoring namenode health on http://{masternode_ip}/50070

However when I deactivate any of the hadoop slave nodes by executing commands:
../sbin/hadoop-daemons.sh stop datanode
../sbin/yarn-daemons.sh stop nodemanager

Namenode heath at http://{masternode_ip}/50070 does not show the deactivation of the slave
node. But I could verify the shut-down of datanode and nodemanager jvm processes by executing
"jps" on the slave node.

Namenode eventually after 20-30 minutes marks the salve node dead.
What am I missing here? Why namenode and resourcemanager are not getting notified of
the datanode and nodemanager deactivation?
Please help, thanks

View raw message