spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shruti Gumma (JIRA)" <>
Subject [jira] [Created] (SPARK-25428) Support plain Kerberos Authentication with Spark
Date Fri, 14 Sep 2018 06:02:00 GMT
Shruti Gumma created SPARK-25428:

             Summary: Support plain Kerberos Authentication with Spark
                 Key: SPARK-25428
             Project: Spark
          Issue Type: New Feature
          Components: Spark Core
    Affects Versions: 2.3.1, 2.3.0, 2.2.2, 2.2.1, 2.2.0, 2.1.3, 2.1.2, 2.1.1
            Reporter: Shruti Gumma

Spark should work with plain Kerberos authentication. Currently, Spark can work with Hadoop
delegation tokens, but not plain Kerberos. Hadoop's UserGroupInformation(UGI) class is responsible
for handling security authentication in Spark. This UserGroupInformation(UGI) has support
for Kerberos authentication, as well as Token authentication. Since Spark does not work correctly
with the Kerberos auth method, it leads to a gap in fully supporting all the security authentication
 If Kerberos is used to login in UserGroupInformation(UGI) using keytabs at the startup of
drivers and executors, then Spark does not allow this logged-in UserGroupInformation(UGI)
user to correctly propagate. The exception arises from the implementation of the runAsSparkUser
method in SparkHadoopUtil.
 The runAsSparkUser method in SparkHadoopUtil creates a new UGI based on the current static
UGI and then transfers credentials from this current static UGI to the new UGI. This works
well with other auth methods, except Kerberos. Transfer credentials implementation is not
conducive for Kerberos auth model since it does not transfer all the required internal state
of UGI( such as isKeytab and isKrbTkt). For Kerberos, the UGI has to be created from UGI.loginUserFromKeytab
method only and not simply by doing a transfer credentials from the previous UGI to the new
 Ideally, the CoarseGrainedExecutorBackend should login using keytab, similar to MesosCoarseGrainedExecutorBackend.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message