hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4457) WebHDFS obtains/sets delegation token service hostname using wrong config leading to issues when NN is configured with RPC IP
Date Mon, 04 Feb 2013 16:02:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13570348#comment-13570348

Daryn Sharp commented on HDFS-4457:

True, NAT probably isn't a problem.  However, most proxies rewrite the Host: header (sometimes
preserve the client's in X-Forwarded-Host: ) or ignore it due to security issues.  The token
comes back with a different authority in the service than the client actually used.  This
is just an example of where the server doesn't know how the client originally tried to contact

For the creds key issue, if you insert a token into the creds with the wrong key, the fs will
try to fetch a token again during job submission.  The creds key typically matches the service,
but doesn't have to, ex. HA tokens contain the logical uri's authority.  If/when webhdfs implements
HA, this will be a problem.  This is why passing credentials obtained via fetchdt is the only
current safe way to ensure tokens are keyed correctly.

All said, my big issue is I specifically ensured in 205+ that clients do not use the server's
service and set the service themselves.  This will allow a backwards compatible change of
the server setting the service to a unique identifier that new clients can use for better
token selection.  If you regress this work by making a client rely on the server setting the
service to the client's notion of the authority, then you will break all hope for a backwards
compatible change...
> WebHDFS obtains/sets delegation token service hostname using wrong config leading to
issues when NN is configured with RPC IP
> -------------------------------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-4457
>                 URL: https://issues.apache.org/jira/browse/HDFS-4457
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 1.1.1, 2.0.2-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Critical
>         Attachments: HDFS_4457.patch, HDFS_4457.patch
> If the NameNode RPC address is configured with an wildcard IP, then delegationotkens
are configured with as service and this breaks clients trying to use those tokens.
> Looking at NamenodeWebHdfsMethods#generateDelegationToken() the problem is SecurityUtil.setTokenService(t,
namenode.getHttpAddress());, tracing back what is being used to resolve getHttpAddress() the
NameNodeHttpServer is resolving the httpAddress doing a httpAddress = new InetSocketAddress(bindAddress.getAddress(),
> , and if using "" in the configuration, you get from bindAddress.getAddress().
> Normally (non webhdfs) this is not an issue because it is the responsibility of the client,
but in the case of WebHDFS, WebHDFS does it before returning the string version of the token
(it must be this way because the client may not be a java client at all and cannot manipulate
the DelegationToken as such).
> The solution (thanks to Eric Sammer for helping figure this out) is for WebHDFS to use
the exacty hostname that came in the HTTP request as the service to set in the delegation

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message