hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dian Fu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6465) Enable the configuration of multiple clusters
Date Mon, 09 Jun 2014 09:27:02 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14021786#comment-14021786
] 

Dian Fu commented on HDFS-6465:
-------------------------------

update some design details about configurations:

requirements:
1.	Existing deployments must be able to use the existing configuration without any change.
2.	As many as possible the configurations for different clusters must be the same. The special
configuration required for different clusters should be minimal.

Configurations added:
•	DFS_REGION_ID("dfs.region.id") : the region id of current cluster
•	DFS_REGIONS("dfs.regions") : the region ids of all clusters, including both the primary
cluster and mirror clusters
•	DFS_REGION_PRIMARY("dfs.region.primary") : the region id of primary cluster

Configurations must be suffixed with regionId:
DFS_NAMENODE_RPC_ADDRESS_KEY, DFS_NAMENODE_SERVICE_RPC_ADDRESS_KEY, DFS_NAMENODE_HTTP_ADDRESS_KEY,
DFS_NAMENODE_HTTPS_ADDRESS_KEY, DFS_NAMENODE_SECONDARY_HTTP_ADDRESS_KEY and DFS_NAMENODE_BACKUP_ADDRESS_KEY


Configurations could be suffixed with regionId or not.
These include all the configurations in NameNode.NAMENODE_SPECIFIC_KEYS and NameNode.NAMESERVICE_SPECIFIC_KEYS
except the above configurations which must be suffixed with regionId:
DFS_NAMENODE_RPC_BIND_HOST_KEY, DFS_NAMENODE_NAME_DIR_KEY, DFS_NAMENODE_EDITS_DIR_KEY, DFS_NAMENODE_SHARED_EDITS_DIR_KEY,
DFS_NAMENODE_CHECKPOINT_DIR_KEY, DFS_NAMENODE_CHECKPOINT_EDITS_DIR_KEY, DFS_NAMENODE_SERVICE_RPC_BIND_HOST_KEY,
DFS_NAMENODE_HTTP_BIND_HOST_KEY,
DFS_NAMENODE_HTTPS_BIND_HOST_KEY, DFS_NAMENODE_KEYTAB_FILE_KEY, DFS_NAMENODE_SECONDARY_HTTPS_ADDRESS_KEY,
DFS_SECONDARY_NAMENODE_KEYTAB_FILE_KEY, DFS_NAMENODE_BACKUP_HTTP_ADDRESS_KEY, DFS_NAMENODE_BACKUP_SERVICE_RPC_ADDRESS_KEY,
DFS_NAMENODE_KERBEROS_PRINCIPAL_KEY, DFS_NAMENODE_KERBEROS_INTERNAL_SPNEGO_PRINCIPAL_KEY,
DFS_HA_FENCE_METHODS_KEY, DFS_HA_ZKFC_PORT_KEY and DFS_HA_AUTO_FAILOVER_ENABLED_KEY
The above configurations can be configured in the following format to distinguish between
clusters:
<configuration key>.<nameservice id>.<namenode id>.<region id>
If a configuration with a region id as suffix cannot be found, the configuration without region
id as suffix will be used instead.

All other configurations which aren’t mentioned should not be suffixed with regionId.

> Enable the configuration of multiple clusters
> ---------------------------------------------
>
>                 Key: HDFS-6465
>                 URL: https://issues.apache.org/jira/browse/HDFS-6465
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>         Attachments: HDFS-6465.1.patch, HDFS-6465.2.patch, HDFS-6465.patch
>
>
> Tracks the changes required for configuration DR.
> configurations added:
>     DFS_REGION_ID("dfs.region.id") : the region id of current cluster
>     DFS_REGIONS("dfs.regions") : the region ids of all clusters, including both the primary
cluster and mirror cluster
>     DFS_REGION_PRIMARY("dfs.region.primary") : the region id of primary cluster
> configurations modified:
>     The configurations in NAMENODE.NAMENODE_SPECIFIC_KEYS can be configured in the following
format to distinguish between clusters.
>     If a configuration with a region id as suffix cannot be found, the configuration
without region id as suffix will be used instead:
>     <configuration key>.<nameservice id>.<namenode id>.<region id>
> The configurations in NAMENODE.NAMESERVICE_SPECIFIC_KEYS can be configured in the following
format to distinguish between clusters.
> If a configuration with a region id as suffix cannot be found, the configuration without
region id as suffix will be used instead:
> <configuration key>.<nameservice id>.<region id>



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message