hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bc Wong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-941) RM Should have a way to update the tokens it has for a running application
Date Tue, 01 Jul 2014 23:04:26 GMT

    [ https://issues.apache.org/jira/browse/YARN-941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14049414#comment-14049414

bc Wong commented on YARN-941:

I'm fine with [~xgong]'s solution. I'd still like to see something more generic to make tokens
(HDFS token, HBase token, etc) work with long running apps though. Perhaps I'll pursue the
"arbitrary expiration time" approach in another jira.

RPC privacy is a very expensive solution for AM-RM communication. First, it needs setup so
AM/RM have access to key infrastructure - having this burden on all applications is not reasonable.
This is compounded by the fact that we use AMRMTokens in non-secure mode too. Second, AM -
RM communication is a very chatty protocol, it's likely the overhead is huge..

True security is often costly. The web/consumer industry went through the same exercise with
HTTP vs HTTPS. You can get at least 10x better performance with HTTP. But in the end, everybody
decided that it's worth it. And passing tokens around without RPC privacy is just like sending
passwords around on HTTP without SSL.

Unfortunately with long running services (the focus of this JIRA), this attack and its success
is not as unlikely. This is the very reason why we roll master-keys every so often in the
first place.

With the rolling master key, it's unlikely for the attack to gather enough cipher text to
mount that attack. Besides, a longer key would require so much computation to attack that
it'd be infeasible.

Anyway, appreciate your response, and I'll follow up in another jira.

> RM Should have a way to update the tokens it has for a running application
> --------------------------------------------------------------------------
>                 Key: YARN-941
>                 URL: https://issues.apache.org/jira/browse/YARN-941
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Robert Joseph Evans
>            Assignee: Xuan Gong
>         Attachments: YARN-941.preview.2.patch, YARN-941.preview.3.patch, YARN-941.preview.4.patch,
> When an application is submitted to the RM it includes with it a set of tokens that the
RM will renew on behalf of the application, that will be passed to the AM when the application
is launched, and will be used when launching the application to access HDFS to download files
on behalf of the application.
> For long lived applications/services these tokens can expire, and then the tokens that
the AM has will be invalid, and the tokens that the RM had will also not work to launch a
new AM.
> We need to provide an API that will allow the RM to replace the current tokens for this
application with a new set.  To avoid any real race issues, I think this API should be something
that the AM calls, so that the client can connect to the AM with a new set of tokens it got
using kerberos, then the AM can inform the RM of the new set of tokens and quickly update
its tokens internally to use these new ones.

This message was sent by Atlassian JIRA

View raw message