hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Kanter (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-15815) Allow to pass some Oozie properties to Spark in HoS
Date Tue, 07 Feb 2017 01:30:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-15815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855116#comment-15855116
] 

Robert Kanter commented on HIVE-15815:
--------------------------------------

[~ctang.ma], I understand wanting to be consistent with how other things are passing properties
to Spark, but why are they using {{spark.hadoop.oozie*}}?  OOZIE-2786 is a related JIRA that
[~zhengxb2005] has been working on for Oozie --> Spark directly, and whichever prefix we
decide to go with, I think we should make this JIRA, which is doing Oozie --> Hive -->
Spark consistent with it.  In OOZIE-2786, we were looking at simply doing {{spark.oozie.*}}.
 

> Allow to pass some Oozie properties to Spark in HoS
> ---------------------------------------------------
>
>                 Key: HIVE-15815
>                 URL: https://issues.apache.org/jira/browse/HIVE-15815
>             Project: Hive
>          Issue Type: Improvement
>          Components: Diagnosability, Spark
>            Reporter: Chaoyu Tang
>            Assignee: Chaoyu Tang
>            Priority: Minor
>         Attachments: HIVE-15815.patch, HIVE-15815.patch
>
>
> Oozie passes some of its properties (e.g. oozie.job.id) to Beeline/HS2 when it invokes
Hive2 action. If we allow these properties to be passed to Spark in HoS, we can easily associate
an Ooize workflow ID to an HoS client and Spark job in Spark history. It will be very helpful
in diagnosing some issues involving Oozie Hive2/HoS/Spark.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message