hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1541) Not marking datanodes dead When namenode in safemode
Date Thu, 17 Mar 2011 23:10:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008211#comment-13008211
] 

Hairong Kuang commented on HDFS-1541:
-------------------------------------

When replication queue starts to populate (if we set the threshold to be small enough), the
traffic on block reports dramatically slows down. That's why I think it is unlikely for NN
to make a wrong decision on dead nodes. Using the time in between block replication time and
safemode exit, NN might be able to catch those datanodes that are really dead in safemode.

But if everybody thinks we should use safemode as the guard instead, I see the benefit too
and I am not against it. Let me upload a new patch.

> Not marking datanodes dead When namenode in safemode
> ----------------------------------------------------
>
>                 Key: HDFS-1541
>                 URL: https://issues.apache.org/jira/browse/HDFS-1541
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.23.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.23.0
>
>         Attachments: deadnodescheck.patch
>
>
> In a big cluster, when namenode starts up,  it takes a long time for namenode to process
block reports from all datanodes. Because heartbeats processing get delayed, some datanodes
are erroneously marked as dead, then later on they have to register again, thus wasting time.
> It would speed up starting time if the checking of dead nodes is disabled when namenode
in safemode.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message