hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-941) RM Should have a way to update the tokens it has for a running application
Date Tue, 01 Jul 2014 00:57:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048384#comment-14048384
] 

Vinod Kumar Vavilapalli commented on YARN-941:
----------------------------------------------

[~bcwalrus],

Your point about us remembering the AMRMTokens is right, I stand corrected.

Responses inline

bq. The attacker gains access to the persistence store, where the RM stores its <id, password>
map. In this case, all bets are off. Neither solution is more secure than the other.
That is correct. This problem is handled by securing the state-store, which we require today.

bq. The attacker snoops an insecure RPC channel and reads valid tokens from the network. The
proper solution is to turn on RPC privacy. The token replacement patch does not offer any
real protection. On the contrary, it may give people a false sense of security, which would
be worse.
RPC privacy is a very expensive solution for AM-RM communication. First, it needs setup so
AM/RM have access to key infrastructure - having this burden on all applications is not reasonable.
This is compounded by the fact that we use AMRMTokens in non-secure mode too. Second, AM -
RM communication is a very chatty protocol, it's likely the overhead is huge..

You are right in that token renewal+replacement is a mitigative solution but the plus is that
it can do so without that much cost.

bq. The attacker mounts a cryptographic attack, or somehow manages to guess a valid <id,
password> pair. Token replacement is better because it limits the exposure. But this attack
is very unlikely. And we can counter that by using a stronger hash function.
Unfortunately with long running services (the focus of this JIRA), this attack and its success
is not as unlikely. This is the very reason why we roll master-keys every so often in the
first place. For long-running services, AMRMTokens play a very similar role of master-keys
between Hadoop daemons.

Overall, I think token replacement is not as complex as you may think it is. The evidence
to that is the redistribution of NMTokens that we *already* do. And as I see Xuan already
has a patch.

Our client, for all our efforts, already is fat. The way we handled the burden of NMTokens
etc is by having a smarter client that takes care of the replacement for users. We can do
the same for AMRMTokens too..

> RM Should have a way to update the tokens it has for a running application
> --------------------------------------------------------------------------
>
>                 Key: YARN-941
>                 URL: https://issues.apache.org/jira/browse/YARN-941
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Robert Joseph Evans
>            Assignee: Xuan Gong
>         Attachments: YARN-941.preview.2.patch, YARN-941.preview.3.patch, YARN-941.preview.4.patch,
YARN-941.preview.patch
>
>
> When an application is submitted to the RM it includes with it a set of tokens that the
RM will renew on behalf of the application, that will be passed to the AM when the application
is launched, and will be used when launching the application to access HDFS to download files
on behalf of the application.
> For long lived applications/services these tokens can expire, and then the tokens that
the AM has will be invalid, and the tokens that the RM had will also not work to launch a
new AM.
> We need to provide an API that will allow the RM to replace the current tokens for this
application with a new set.  To avoid any real race issues, I think this API should be something
that the AM calls, so that the client can connect to the AM with a new set of tokens it got
using kerberos, then the AM can inform the RM of the new set of tokens and quickly update
its tokens internally to use these new ones.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message