hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3053) [Security] Review and implement security in ATS v.2
Date Wed, 16 Nov 2016 19:16:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-3053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15671338#comment-15671338
] 

Varun Saxena commented on YARN-3053:
------------------------------------

Well the reason I was talking about fetching delegation token was because of the support in
terms of web/HTTP filters in current hadoop common code.
Refer to DelegationTokenAuthenticationFilter and DelegationTokenAuthenticatedURL. Delegation
token allows AM to post entities to timeline service if it cannot do kerberos login in a secure
setup. Also we can renew these tokens.
This is what we do in ATSv1 too.
As Timeline Client is web based, we can embed the delegation token and use it for authentication.
The filter I talked about above will automatically intercept the token and verify the token
using backing token secret manager. When I said that collector can generate a delegation token
and send it back to NM, I meant we can use the same secret manager to generate the token which
the filter uses so that when AM sends it in HTTP request, its available with collector manager.
Is there any problem with it ?

Now the issue with timeline service v2 is that we do not know where the collector is running
and collector won't start till we send the request to a NM to launch AM. Now if we want tokens
to be set in ApplicationSubmissionContext while launching AM, we cannot really do it when
collector is launched. RM/NM (approach 2 or 3 in doc) will have to generate a token per app
on behalf of collector before it launches and then inform the collector about this token.
But here, tokens typically use sequence numbers and they may clash and master keys used in
both the processes(if collector is running outside NM) will have to be in sync as well.

If the solution suggested i.e. collector generating a token explicitly when it starts and
sending it back to NM is not agreed upon, one other option would be for RM or NM to explicitly
contact timeline service to get a delegation token when a new timeline service address is
reported to it. This would be similar to what we do for ATSv1 at client side during job submission
except that we will have to use RM/NM login user for authentication. 
One more issue is who renews these tokens and cancels them. Probably collector manager can
do it by itself as it is informed when app finishes. We will have to handle failover scenarios
as well i.e. store all the tokens in state store.

> [Security] Review and implement security in ATS v.2
> ---------------------------------------------------
>
>                 Key: YARN-3053
>                 URL: https://issues.apache.org/jira/browse/YARN-3053
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Varun Saxena
>              Labels: YARN-5355
>         Attachments: ATSv2Authentication(draft).pdf
>
>
> Per design in YARN-2928, we want to evaluate and review the system for security, and
ensure proper security in the system.
> This includes proper authentication, token management, access control, and any other
relevant security aspects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message