spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vanzin <...@git.apache.org>
Subject [GitHub] spark pull request #14065: [SPARK-14743][YARN] Add a configurable credential...
Date Tue, 09 Aug 2016 17:26:45 GMT
Github user vanzin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/14065#discussion_r74104234
  
    --- Diff: docs/running-on-yarn.md ---
    @@ -525,16 +524,23 @@ token for the cluster's HDFS filesystem, and potentially for HBase
and Hive.
     
     An HBase token will be obtained if HBase is in on classpath, the HBase configuration
declares
     the application is secure (i.e. `hbase-site.xml` sets `hbase.security.authentication`
to `kerberos`),
    -and `spark.yarn.security.tokens.hbase.enabled` is not set to `false`.
    +and `spark.yarn.security.credentials.hbase.enabled` is not set to `false`.
     
     Similarly, a Hive token will be obtained if Hive is on the classpath, its configuration
     includes a URI of the metadata store in `"hive.metastore.uris`, and
    -`spark.yarn.security.tokens.hive.enabled` is not set to `false`.
    +`spark.yarn.security.credentials.hive.enabled` is not set to `false`.
     
     If an application needs to interact with other secure HDFS clusters, then
     the tokens needed to access these clusters must be explicitly requested at
     launch time. This is done by listing them in the `spark.yarn.access.namenodes` property.
     
    +Spark supports integrating with other security-aware services through Java Services mechanism
(see
    +`java.util.ServiceLoader`). To do that, implementations of `org.apache.spark.deploy.yarn.security.ServiceCredentialProvider`
    +should be available to Spark by listing their names in the corresponding file in the
jar's
    +`META-INF/services` directory. These plug-ins can be disabled by setting
    +`spark.yarn.security.tokens.{service}.enabled` to `false`, where `{service}` is the name
of
    +credential provider.
    +
     ```
     spark.yarn.access.namenodes hdfs://ireland.example.org:8020/,hdfs://frankfurt.example.org:8020/
    --- End diff --
    
    Hmm, it seems like this example should be before the paragraph you're adding.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message