hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yibing Shi (JIRA)" <>
Subject [jira] [Commented] (HIVE-15767) Hive On Spark is not working on secure clusters from Oozie
Date Thu, 13 Jul 2017 12:08:00 GMT


Yibing Shi commented on HIVE-15767:

bq. I think the Spark driver will get the tokens afterwards
I really doubt that Spark driver can do this. In Oozie environment, it is Oozie server that
obtains all the tokens *on behalf of the end user*. When the Hive actions starts a Spark job,
the Spark driver has no access to end user ticket or keytab file. I don't think it can obtain
necessary tokens. 
I believe we should somehow extract all the tokens from existing toke file, and pass it on
to the Spark driver.

> Hive On Spark is not working on secure clusters from Oozie
> ----------------------------------------------------------
>                 Key: HIVE-15767
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.1, 2.1.1
>            Reporter: Peter Cseh
>            Assignee: Peter Cseh
>         Attachments: HIVE-15767-001.patch, HIVE-15767-002.patch
> When a HiveAction is launched form Oozie with Hive On Spark enabled, we're getting errors:
> {noformat}
> Caused by: Exception reading file:/yarn/nm/usercache/yshi/appcache/application_1485271416004_0022/container_1485271416004_0022_01_000002/container_tokens
>         at
>         at
> {noformat}
> This is caused by passing the {{mapreduce.job.credentials.binary}} property to the Spark
configuration in RemoteHiveSparkClient.

This message was sent by Atlassian JIRA

View raw message