hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7374) Allow decommissioning of dead DataNodes
Date Fri, 14 Nov 2014 20:39:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14212794#comment-14212794
] 

Ming Ma commented on HDFS-7374:
-------------------------------

[~andrew.wang], after a node is dead, all its blocks will be removed from blockmap. So if
the node no longer joins the cluster, it isn't unclear how you can tell if all its blocks
are fully replicated unless we track those blocks.

Another way to cover all these scenarios could be to get rid of {{DEAD, DECOM_IN_PROGRESS}}
state. After the node is dead during decommission, transition to {{DEAD, DECOMMED}}. When
the node rejoins the cluster, transition it to {{LIVE, DECOM_IN_PROGRESS}}.

> Allow decommissioning of dead DataNodes
> ---------------------------------------
>
>                 Key: HDFS-7374
>                 URL: https://issues.apache.org/jira/browse/HDFS-7374
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-7374-001.patch, HDFS-7374-002.patch
>
>
> We have seen the use case of decommissioning DataNodes that are already dead or unresponsive,
and not expected to rejoin the cluster.
> The logic introduced by HDFS-6791 will mark those nodes as {{DECOMMISSION_INPROGRESS}},
with a hope that they can come back and finish the decommission work. If an upper layer application
is monitoring the decommissioning progress, it will hang forever.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message