hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-993) Namenode does not need to store any data node info persistently.
Date Fri, 16 Mar 2007 17:48:09 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12481712

dhruba borthakur commented on HADOOP-993:

I would say that the the motivation behind this is to make the architecture cleaner and simple.
The filesystem image need not store cluster configuration (i.e. datanodes). Regarding you

* "in case of a misconfiguration of a datanode, namenode can tell which datanode "should"
have a certain storageid" -- You might be right here. But from my understanding, we have never
had exploited this particular feature on our kryptonite cluster till now.
* "lastly - UI can report on missing datanodes, which proved immensely useful for administration"
-- The preferred method to maintain cluster membership info is in the include/exclude files.
The include/exclude file support was introduced recently.

I think it is architecturally sound to keep cluster membership info separate from filesystem
namespace information. It might help in clustering the namenode (thru namespace partitioning)
or in namenode failover scenarios in the future.

I agree with you that the cluster administrator will not be able to see dead data nodes in
the UI anymore. This seems to be the biggest drawback of this patch. If the adminstrator wants
to view all datanodes in the UI, he/she has to create a include file (manually) and list all
datanodes in that file. Do you agree that this alternative is acceptable?


> Namenode does not need to store any data node info persistently.
> ----------------------------------------------------------------
>                 Key: HADOOP-993
>                 URL: https://issues.apache.org/jira/browse/HADOOP-993
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Raghu Angadi
>         Assigned To: Sameer Paranjpye
>         Attachments: noDatanodesInFsimage.patch
> Namenode does not need to serialize datanode info. It will map datanode to storageID
when datanode register.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message