hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Xin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12951) Reduce Spark executor prewarm timeout to 5s
Date Wed, 26 Oct 2016 02:30:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15607154#comment-15607154
] 

Richard Xin commented on HIVE-12951:
------------------------------------

Thanks Rui and Xuefu for your quick replies.

>> The right comparison is to measure the performance with and without container prewarming
to see if there is any benefit for your first run
we didn't feel any performance benefit with prewarming for the first run. I assume that "first
run" here refers to first run per hive session, instead of first run after hive restarts.
We are trying to build a web service to run ad-hoc queries against hive, so each call could
be the "first call" for the session.  do you have any recommendations on how we should implement
the solution? Based to our tests, some sessions' first query under hive on spark is slower
than hive on mr. 
Thanks in advance for your insight.


> Reduce Spark executor prewarm timeout to 5s
> -------------------------------------------
>
>                 Key: HIVE-12951
>                 URL: https://issues.apache.org/jira/browse/HIVE-12951
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>             Fix For: 2.1.0
>
>         Attachments: HIVE-12951.patch, HIVE-12951.patch
>
>
> Currently it's set to 30s, which tends to be longer than needed. Reduce it to 5s, only
considering jvm startup time. (Eventually, we may want to make this configurable.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message