hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lefty Leverenz (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11363) Prewarm Hive on Spark containers [Spark Branch]
Date Mon, 03 Aug 2015 05:31:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14651462#comment-14651462
] 

Lefty Leverenz commented on HIVE-11363:
---------------------------------------

This is fixed by HIVE-11434 (Followup for HIVE-10166: reuse existing configurations for prewarming
Spark executors).

> Prewarm Hive on Spark containers [Spark Branch]
> -----------------------------------------------
>
>                 Key: HIVE-11363
>                 URL: https://issues.apache.org/jira/browse/HIVE-11363
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>    Affects Versions: 1.1.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>              Labels: TODOC-SPARK, TODOC1.3
>             Fix For: spark-branch, 1.3.0, 2.0.0
>
>         Attachments: HIVE-11363.1-spark.patch, HIVE-11363.2-spark.patch, HIVE-11363.3-spark.patch,
HIVE-11363.4-spark.patch, HIVE-11363.5-spark.patch
>
>
> When Hive job is launched by Oozie, a Hive session is created and job script is executed.
Session is closed when Hive job is completed. Thus, Hive session is not shared among Hive
jobs either in an Oozie workflow or across workflows. Since the parallelism of a Hive job
executed on Spark is impacted by the available executors, such Hive jobs will suffer the executor
ramp-up overhead. The idea here is to wait a bit so that enough executors can come up before
a job can be executed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message