hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4457) WebHDFS obtains/sets delegation token service hostname using wrong config leading to issues when NN is configured with RPC IP
Date Thu, 31 Jan 2013 21:15:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568111#comment-13568111

Alejandro Abdelnur commented on HDFS-4457:


Regarding "The client should use the service as set by the server,", this is not true today,
DFSClient.getDelegationToken() & WebHdfsFileSystem.getDelegationToken() sets the service
HOST:PORT when it receives the token using the HOST:PORT used by the client. The token fetched
by a client should have the HOST:PORT the client used to access the service (HDFS/WEBHDFS),
if the service is being offered with multihoming, the service cannot know/assume what HOST
are visible to the client, unless the it has a hint from the client (this is what this patch
is done for WebHDFS).

Regarding "You can't assume...", I don't think there is any assumption here, the client is
getting a blob and passes that blob to another client, that is the whole purpose of delegation
tokens, no? This is using the built in mechanism in MapReduce to populate the token cache
with the contents of the binary representation of the token. This ends up being used to submit
the job, thus in the credentials of the alternate client.

What this patch does, is to enable non java clients that request a delegation token from WebHDFS
to have a delegation token ready to be handed to another client. 

This patch does not affect in any maner java clients that request a delegation token as they
continue injecting the service when they receive the delegation token. 

Oozie does not have this problem because Oozie uses the java client and gets the service injection
for free. Hue in the other hand, because not being Java, cannot do this.

Regarding fetchdt, it does not seem to fetch tokens of WebHDFS.

Finally, in the scenario the approach I'm proposing does not cut it, we still have problem
with WebHDFS using setting the IP wildcard of the NN RPC configuration.


> WebHDFS obtains/sets delegation token service hostname using wrong config leading to
issues when NN is configured with RPC IP
> -------------------------------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-4457
>                 URL: https://issues.apache.org/jira/browse/HDFS-4457
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 1.1.1, 2.0.2-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Critical
>         Attachments: HDFS_4457.patch, HDFS_4457.patch
> If the NameNode RPC address is configured with an wildcard IP, then delegationotkens
are configured with as service and this breaks clients trying to use those tokens.
> Looking at NamenodeWebHdfsMethods#generateDelegationToken() the problem is SecurityUtil.setTokenService(t,
namenode.getHttpAddress());, tracing back what is being used to resolve getHttpAddress() the
NameNodeHttpServer is resolving the httpAddress doing a httpAddress = new InetSocketAddress(bindAddress.getAddress(),
> , and if using "" in the configuration, you get from bindAddress.getAddress().
> Normally (non webhdfs) this is not an issue because it is the responsibility of the client,
but in the case of WebHDFS, WebHDFS does it before returning the string version of the token
(it must be this way because the client may not be a java client at all and cannot manipulate
the DelegationToken as such).
> The solution (thanks to Eric Sammer for helping figure this out) is for WebHDFS to use
the exacty hostname that came in the HTTP request as the service to set in the delegation

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message