hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: dfs datanode heartbeats and getBlockwork requests
Date Mon, 03 Apr 2006 21:42:20 GMT
Hairong Kuang wrote:
> Currently dfs datanodes send heartbeats and getBlockwork requests to the
> namenode at the same frequency (once every 3 seconds) after certain startup
> time. Is there any design reason that we need two seperate messages instead
> of one? I am thinking that if we let a sendHeartbeat request return the
> blocks to be deleted or replicated, we are able to cut the network traffic
> in dfs.

No, that sounds like a reasonable change to me.

The startup delay will be need to be somehow re-implemented.  Perhaps we 
could simply change this to a timer in the namenode on startup, so that 
it waits a while on startup before giving any blockwork.  We might then 
have issues if, e.g, the namenode's ethernet cable were yanked for a few 
minutes.  When it is re-connected, the namenode will start issuing lots 
of uneeded replication requests.  Having a delay in blockwork at the 
datanode each time it establishes a new connection to the namenode 
solves that problem.  Are there other cases that the current startup 
blockwork delay is handling?


View raw message