spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giovanni Lanzani (JIRA)" <>
Subject [jira] [Commented] (SPARK-9042) Spark SQL incompatibility with Apache Sentry
Date Fri, 28 Aug 2015 13:34:45 GMT


Giovanni Lanzani commented on SPARK-9042:

If this is ironic, no I don't have an idea. If it's not, then no, I don't have an idea.

> Spark SQL incompatibility with Apache Sentry
> --------------------------------------------
>                 Key: SPARK-9042
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.2.0
>            Reporter: Nitin Kak
> Hive queries executed from Spark using HiveContext use CLI to create the query plan and
then access the Hive table directories(under /user/hive/warehouse/) directly. This gives AccessContolException
if Apache Sentry is installed:
> Permission denied: user=kakn, access=READ_EXECUTE,
> With Apache Sentry, only "hive" user(created only for Sentry) has the permissions to
access the hive warehouse directory. After Sentry installations all the queries are directed
to HiveServer2 which translates the changes the invoking user to "hive" and then access the
hive warehouse directory. However, HiveContext does not execute the query through HiveServer2
which is leading to the issue. Here is an example of executing hive query through HiveContext.
> val hqlContext = new HiveContext(sc) // Create context to run Hive queries 
> val pairRDD = hqlContext.sql(hql) // where hql is the string with hive query 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message