hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3153) For HA, a logical name is visible in URIs - add an explicit logical name
Date Sat, 01 Dec 2012 00:01:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507775#comment-13507775

Sanjay Radia commented on HDFS-3153:

Updated propose config
dfs.namespace.volumes= v1,v2                     // two volumes in the cluster
dfs.namespace.volume.v1.hdfs    = v1host:rpc_port   // host:port for scheme hdfs for namespace
dfs.namespace.volume.v1.hftp    = v1host:http_port  // host:port for scheme hftp for namespace
dfs.namespace.volume.v1.webhdfs = v1host:http_port  // host:port for scheme webhdfs  for namespace

dfs.namespace.volume.v2.hdfs    = v2host:rpc_port   // Same for volume v2
dfs.namespace.volume.v2.hftp    = v2host:http_port
dfs.namespace.volume.v2.webhdfs = v2host:http_port
Then, the logical namespaces are mapped to physical hosts during FileSystem initialization
as follows:
hdfs://v1/path      => hdfs://v1host:v1rpc_port/path
hdfs://v1:port/path => hdfs://v1host:port/path
hftp://v1/path      => hftp://v1host:v1http_port/path
webhdfs://v1/path   => webhdfs://v1host:v1http_port/path

For HA with IP failvoer, the above suffices - there is single IP that fails over.
For HA with dual IPs the configuration one specifies multiple ips:
dfs.namespace.volume.v1.hdfs    = v1host1:rpc_port, v1host2:rpc_port
dfs.namespace.volume.v1.hftp    = v1host1:http_port, v1host2:http_port
dfs.namespace.volume.v1.webhdfs = v1host1:http_port, v1host2:http_port

dfs.namespace.volume.v2.hdfs    = v2host1:rpc_port, v2host2:rpc_port
dfs.namespace.volume.v2.hftp    = v2host1:http_port, v2host2:http_port
dfs.namespace.volume.v2.webhdfs = v2host1:http_port, v2host2:http_port

We allow separate namenode service rpc host-ports for DNs, Secondary to communicate.
These can be specified as follows:
dfs.namespace.volume.v1.service-rpc    = v1host:rpc_port
dfs.namespace.volume.v2.service-rpc    = v1host:rpc_port

The following new config names introduced in Hadoop 2 would be removed:
dfs.nameservices   // replaced by dfs.namespaces.volumes
dfs.ha.nodenodes.v1=nn1,nn2   // replaced by dfs.namespace.v1.hdfs=host:port,host2:port

> For HA, a logical name is visible in URIs - add an explicit logical name
> ------------------------------------------------------------------------
>                 Key: HDFS-3153
>                 URL: https://issues.apache.org/jira/browse/HDFS-3153
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Sanjay Radia
>            Assignee: Tsz Wo (Nicholas), SZE
>            Priority: Critical

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message