hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9005) Provide support for upgrade domain script
Date Thu, 10 Sep 2015 19:07:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739386#comment-14739386

Colin Patrick McCabe commented on HDFS-9005:

Yes, a refresh command would be nice.  This seems kind of similar to the datanode include
/ exclude lists.

Hmm.  What if we replaced the current include and exclude files with a single JSON configuration
file like this:
  "included": [
      "hostname" : "a2402.example.com",
      "upgradeDomain" : "domain1"
      "hostname" : "a2404.example.com",
      "upgradeDomain" : "domain1"
      "hostname" : "a2406.example.com",
      "upgradeDomain" : "domain2"
  "excluded": [
      "hostname" : "a2408.example.com",
    }, {
      "hostname" : "a2410.example.com",

That would make it easy to see what was going on with the datanodes in the cluster at a glance.
 It would also allow us to use the existing {{refreshDatanodes}} command to refresh upgrade
domains.  We could use Jackson to do the JSON parsing without writing any code.

> Provide support for upgrade domain script
> -----------------------------------------
>                 Key: HDFS-9005
>                 URL: https://issues.apache.org/jira/browse/HDFS-9005
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Ming Ma
> As part of the upgrade domain feature, we need to provide a mechanism to specify upgrade
domain for each datanode. One way to accomplish that is to allow admins specify an upgrade
domain script that takes DN ip or hostname as input and return the upgrade domain. Then namenode
will use it at run time to set {{DatanodeInfo}}'s upgrade domain string. The configuration
can be something like:
> {noformat}
> <property>
> <name>dfs.namenode.upgrade.domain.script.file.name</name>
> <value>/etc/hadoop/conf/upgrade-domain.sh</value>
> </property>
> {noformat}
> just like topology script, 

This message was sent by Atlassian JIRA

View raw message