hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nauroth <cnaur...@hortonworks.com>
Subject Re: Time until a datanode is marked as dead
Date Fri, 23 Jan 2015 18:23:51 GMT
Hi Frank,

The time period for determining if a datanode is dead is calculated as a
function of a few different configuration properties.  The current
implementation in DatanodeManager.java does it like this:

    final long heartbeatIntervalSeconds = conf.getLong(
        DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY,
        DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_DEFAULT);
    final int heartbeatRecheckInterval = conf.getInt(
        DFSConfigKeys.DFS_NAMENODE_HEARTBEAT_RECHECK_INTERVAL_KEY,
        DFSConfigKeys.DFS_NAMENODE_HEARTBEAT_RECHECK_INTERVAL_DEFAULT); //
5 minutes
    this.heartbeatExpireInterval = 2 * heartbeatRecheckInterval
        + 10 * 1000 * heartbeatIntervalSeconds;

Under default configuration, dfs.namenode.heartbeat.recheck-interval is 5
minutes and dfs.heartbeat.interval is 3 seconds.  If we plug those values
into the formula, we get 10.5 minutes, which agrees with your observation.
If you change dfs.namenode.heartbeat.recheck-interval to 2.5 minutes, then
you'll achieve an effective timeout of 5.5 minutes before a datanode is
marked dead.

dfs.namenode.heartbeat.recheck-interval is not documented in
hdfs-default.xml, though I don't recall if that's an intentional choice or
just an oversight.  The value of the property must be expressed in
milliseconds.

Chris Nauroth
Hortonworks
http://hortonworks.com/


On Thu, Jan 22, 2015 at 11:19 PM, Frank Lanitz <frank.lanitz@sql-ag.de>
wrote:

> Hi,
>
> I'm trying to configure the time a datanode needs to be considered dead.
> Currently it appears to be set to something about 10min which is a
> little to high for my scenario. As I wasn't able to find some obvious
> flag, I've tried to set some properties, which might could do that.
> Without succes. So e.g. I've put into my hdfs-site.xml
>
> <property>
>     <name>dfs.namenode.check.stale.datanode</name>
>     <value>true</value>
>     <description>Activate stale check</description>
> </property>
>
> <property>
>     <name>dfs.namenode.stale.datanode.interval</name>
>     <value>10</value>
>     <description>Timeout</description>
> </property>
>
> So my question is: Which option(s) I have to set in order to e.g.
> decrease time needed to mark a datanode as dead to 5min running 2.6.
>
> Cheers,
> Frank
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message