spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-13148) support zero-keytab Oozie application launch on a secure cluster
Date Tue, 02 Feb 2016 22:06:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-13148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15129136#comment-15129136
] 

Steve Loughran commented on SPARK-13148:
----------------------------------------

Note that Hadoop's UGI class automatically loads the file referenced off {{$HADOOP_TOKEN_FILE_LOCATION}}
when it inits; this is the mechanism used to get tokens in the YARN AM.

Client-side, they become the tokens of the current user. All that is needed is for the Yarn
client to recognise that the situation has occurred (i.e. the env variable is set), add all
those credentials to the AM's launch context —and skip trying to acquire tokens for filesystems,
HBase and Hive.

> support zero-keytab Oozie application launch on a secure cluster 
> -----------------------------------------------------------------
>
>                 Key: SPARK-13148
>                 URL: https://issues.apache.org/jira/browse/SPARK-13148
>             Project: Spark
>          Issue Type: New Feature
>          Components: YARN
>    Affects Versions: 1.6.0
>         Environment: YARN cluster with Kerberos enabled, launched from Oozie —where
Oozie passes down the delegation tokens
>            Reporter: Steve Loughran
>
> Oozie can launch Spark instances on insecure clusters, and on a secure cluster if Oozie
is set up to provide a keytab.
> What it cannot currently do is launch a Spark application on a YARN cluster without a
keytab. In this situation, Oozie collects the delegation tokens it is setup to collect (as
a superuser in the cluster), saves them to a file, then points to the file in the `HADOOP_TOKEN_FILE_LOCATION`
environment variable.
> These tokens need to be used to launch the application —rather than try to get some
more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message