hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7893) Find a way to get a job identifier when submitting a spark job [Spark Branch]
Date Tue, 14 Oct 2014 01:39:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170353#comment-14170353
] 

Rui Li commented on HIVE-7893:
------------------------------

Thanks [~joshrosen]. Yes I've seen your PR and really appreciate your help!
FYI, we're using the async API in HIVE-7439. It's the original foreachAsync I added, of course.
But we can update to use your stabilized ones when your PR gets merged.
Thanks again!

> Find a way to get a job identifier when submitting a spark job [Spark Branch]
> -----------------------------------------------------------------------------
>
>                 Key: HIVE-7893
>                 URL: https://issues.apache.org/jira/browse/HIVE-7893
>             Project: Hive
>          Issue Type: Task
>          Components: Spark
>            Reporter: Rui Li
>            Assignee: Rui Li
>            Priority: Minor
>              Labels: Spark-M3
>
> Currently we use the {{foreach}} RDD action to submit a spark job. In order to implement
job monitoring functionality (HIVE-7438), we need to get a job identifier when submitting
the job, so that we can later register some listener for that specific job.
> This task requires facilitation from spark side (SPARK-2636). I've tried to use {{AsyncRDDActions}}
instead of the traditional actions, and it proved to be a possible way to get the job ID we
need.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message