hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9005) Provide support for upgrade domain script
Date Wed, 06 Jan 2016 06:23:39 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ming Ma updated HDFS-9005:
    Attachment: HDFS-9005.patch

[~ctrezzo] will create a new jira for the design around "passing datanode admins properties
via ClientProtocol" approach. Meanwhile, we should go ahead and use the existing refreshNodes
flow to provide upgrade domain support; also some of the components such as json and datanode
admin data structure can be reused later in the long-term design. Here is the draft patch
that allows admins to specify all DN properties in one json file.  Example:

 {"hostName": "host1"}
 {"hostName": "host2", "port": 50, "upgradeDomain": "ud0"}
 {"hostName": "host3", "port": 0, "adminState": "DECOMMISSIONED"}

* Abstract host file manager into an interface for both the new json approach and the old
include/exclude approach to implement.
* Implement json file manager. The default is still the old include/exclude approach.
* Have {{DatanodeManager}} get the upgrade domain from the host file manager.
* Refactor test code w.r.t. host file writing.

> Provide support for upgrade domain script
> -----------------------------------------
>                 Key: HDFS-9005
>                 URL: https://issues.apache.org/jira/browse/HDFS-9005
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ming Ma
>         Attachments: HDFS-9005.patch
> As part of the upgrade domain feature, we need to provide a mechanism to specify upgrade
domain for each datanode. One way to accomplish that is to allow admins specify an upgrade
domain script that takes DN ip or hostname as input and return the upgrade domain. Then namenode
will use it at run time to set {{DatanodeInfo}}'s upgrade domain string. The configuration
can be something like:
> {noformat}
> <property>
> <name>dfs.namenode.upgrade.domain.script.file.name</name>
> <value>/etc/hadoop/conf/upgrade-domain.sh</value>
> </property>
> {noformat}
> just like topology script, 

This message was sent by Atlassian JIRA

View raw message