hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2450) Only complete hostname is supported to access data via hdfs://
Date Fri, 14 Oct 2011 23:06:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13127965#comment-13127965
] 

Daryn Sharp commented on HDFS-2450:
-----------------------------------

The exception originates in {{FileSystem.checkPath()}} which is used to see if a {{Path}}
belongs to the {{FileSystem}}.  Among other things, it does a simple string comparisons of
the host in the uri's authority.  There is an interaction when setting use_ip=false to enable
the use of host-based tokens.  The {{NetUtils}} methods to create an {{InetSocketAddress}}
will return FQDNs to increase the security of host-based tokens.  With a FQDN, there is no
ambiguity, regardless of the host's dns search path.

The observed issue is that {{DistributedFileSystem}} does not store its designated uri as-is.
 Instead it takes the uri's authority, converts it to a {{InetSocketAddress}}, and then converts
it back to a string.  With use_ip=true, the default behavior, the conversion are unnecessary
but always yield the original value.  With use_ip=false, the host will become a FQDN after
the conversions, so simple string comparison between the FQDN and short hostname fails...

                
> Only complete hostname is supported to access data via hdfs://
> --------------------------------------------------------------
>
>                 Key: HDFS-2450
>                 URL: https://issues.apache.org/jira/browse/HDFS-2450
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.20.205.0
>            Reporter: Rajit Saha
>            Assignee: Daryn Sharp
>
> If my complete hostname is  host1.abc.xyz.com, only complete hostname must be used to
access data via hdfs://
> I am running following in .20.205 Client to get data from .20.205 NN (host1)
> $hadoop dfs -copyFromLocal /etc/passwd  hdfs://host1/tmp
> copyFromLocal: Wrong FS: hdfs://host1/tmp, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd  hdfs://host1.abc/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> $hadoop dfs -copyFromLocal /etc/passwd  hftp://host1.abc.xyz/tmp/
> copyFromLocal: Wrong FS: hdfs://host1.blue/tmp/1, expected: hdfs://host1.abc.xyz.com
> Usage: java FsShell [-copyFromLocal <localsrc> ... <dst>]
> Only following is supported 
> $hadoop dfs -copyFromLocal /etc/passwd  hdfs://host1.abc.xyz.com/tmp/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message