hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-7510) Tokens should use original hostname provided instead of ip
Date Thu, 08 Sep 2011 23:50:09 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100799#comment-13100799

Jitendra Nath Pandey commented on HADOOP-7510:

> How about if I add a config option to control whether the new behavior in this patch
is enabled.
This sounds ok. hostname should be added to token-service only if config is enabled. It should
be off by default. Those who don't want ip-failover will not run into any surprises.

1) mapreduce.job.hdfs-servers is indeed used in this use-case, and that is how the delegation
token for all the namenodes involved will be obtained by the jobClient when first job is submitted.
But, the problem is still there because the first jobClient is a new version client and it
puts hostname in the token. When a task of this first job tries to submit another job to a
cluster of earlier version, it can't get a new token issued because it doesn't have any kerberos
credentials for the user. The existing tokens must be passed along.
2) InetSocketAddress#getHostname javadoc states that it "returns the hostname part of the
address." It doesn't say it would be the same name/ip with which it was instantiated. In fact
java 7 document states that it may trigger a reverse lookup. getHostString looks like a new
API added in java 7.
3) I agree those changes have risks too.

> I'm not sure I understand why you think I'm not using standard java apis. The calls are
all documented and the 
> behavior is standard. I'm open to other means of determining exactly what string was
used to instantiate the 
> InetSocketAddress.
 All I meant was that the assumption about the API may not work in all java implementations.
In Java 7, getHostString does guarantee no reverse lookup. Unfortunately, that API is not
in java 6.

> Tokens should use original hostname provided instead of ip
> ----------------------------------------------------------
>                 Key: HADOOP-7510
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7510
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For:
>         Attachments: HADOOP-7510-2.patch, HADOOP-7510-3.patch, HADOOP-7510-4.patch, HADOOP-7510.patch
> Tokens currently store the ip:port of the remote server.  This precludes tokens from
being used after a host's ip is changed.  Tokens should store the hostname used to make the
RPC connection.  This will enable new processes to use their existing tokens.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message