spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Reynold Xin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-9042) Spark SQL incompatibility with Apache Sentry
Date Fri, 28 Aug 2015 00:04:45 GMT

    [ https://issues.apache.org/jira/browse/SPARK-9042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14717804#comment-14717804
] 

Reynold Xin commented on SPARK-9042:
------------------------------------

[~lanzani] any idea how complicated it is to fix this?


> Spark SQL incompatibility with Apache Sentry
> --------------------------------------------
>
>                 Key: SPARK-9042
>                 URL: https://issues.apache.org/jira/browse/SPARK-9042
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.2.0
>            Reporter: Nitin Kak
>
> Hive queries executed from Spark using HiveContext use CLI to create the query plan and
then access the Hive table directories(under /user/hive/warehouse/) directly. This gives AccessContolException
if Apache Sentry is installed:
> org.apache.hadoop.security.AccessControlException: Permission denied: user=kakn, access=READ_EXECUTE,
inode="/user/hive/warehouse/mastering.db/sample_table":hive:hive:drwxrwx--t 
> With Apache Sentry, only "hive" user(created only for Sentry) has the permissions to
access the hive warehouse directory. After Sentry installations all the queries are directed
to HiveServer2 which translates the changes the invoking user to "hive" and then access the
hive warehouse directory. However, HiveContext does not execute the query through HiveServer2
which is leading to the issue. Here is an example of executing hive query through HiveContext.
> val hqlContext = new HiveContext(sc) // Create context to run Hive queries 
> val pairRDD = hqlContext.sql(hql) // where hql is the string with hive query 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message