hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3153) For HA, a logical name is visible in URIs - add an explicit logical name
Date Thu, 21 Jun 2012 00:23:42 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398025#comment-13398025

Tsz Wo (Nicholas), SZE commented on HDFS-3153:

Here is a proposal.

A volume name is first resolved with conf and then DNS.  It could be resolved into one or
more hostnames.  Multiple hostnames is used in HA setup.

Consider the URI hdfs://V/, where V is a volume name.  If V is a volume name in conf, resolve
it with conf and then resolve it with DNS.  Otherwise, resolve it directly with DNS.  

Consider the following conf setup:
dfs.namespace.volume.<v1>.hdfs    = v1host:rpc_port
dfs.namespace.volume.<v1>.hftp    = v1host:http_port
dfs.namespace.volume.<v1>.webhdfs = v1host:http_port

dfs.namespace.volume.<v2>.hdfs    = v2host:rpc_port
dfs.namespace.volume.<v2>.hftp    = v2host:http_port
dfs.namespace.volume.<v2>.webhdfs = v2host:http_port
Then, the logical namespaces are mapped to physical hosts during FileSystem initialization
as follows:
hdfs://v1/path      => hdfs://v1host:v1rpc_port/path
hdfs://v1:port/path => hdfs://v1host:port/path
hftp://v1/path      => hftp://v1host:v1http_port/path
webhdfs://v1/path   => webhdfs://v1host:v1http_port/path
For HA, the configuration becomes:
dfs.namespace.volume.<v1>.hdfs    = v1host1:rpc_port, v1host2:rpc_port
dfs.namespace.volume.<v1>.hftp    = v1host1:http_port, v1host2:http_port
dfs.namespace.volume.<v1>.webhdfs = v1host1:http_port, v1host2:http_port

dfs.namespace.volume.<v2>.hdfs    = v2host1:rpc_port, v2host2:rpc_port
dfs.namespace.volume.<v2>.hftp    = v2host1:http_port, v2host2:http_port
dfs.namespace.volume.<v2>.webhdfs = v2host1:http_port, v2host2:http_port
> For HA, a logical name is visible in URIs - add an explicit logical name
> ------------------------------------------------------------------------
>                 Key: HDFS-3153
>                 URL: https://issues.apache.org/jira/browse/HDFS-3153
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Sanjay Radia
>            Assignee: Tsz Wo (Nicholas), SZE
>            Priority: Critical

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message