hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12951) Reduce Spark executor prewarm timeout to 5s
Date Tue, 25 Oct 2016 16:20:58 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15605750#comment-15605750
] 

Xuefu Zhang commented on HIVE-12951:
------------------------------------

Besides more containers that are already active, the second run also benefits a lot from JIT
code generation which cannot be done by prewarning containers.

The right comparison is to measure the performance with and without container prewarming to
see if there is any benefit for your first run and make your decision accordingly.

> Reduce Spark executor prewarm timeout to 5s
> -------------------------------------------
>
>                 Key: HIVE-12951
>                 URL: https://issues.apache.org/jira/browse/HIVE-12951
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>             Fix For: 2.1.0
>
>         Attachments: HIVE-12951.patch, HIVE-12951.patch
>
>
> Currently it's set to 30s, which tends to be longer than needed. Reduce it to 5s, only
considering jvm startup time. (Eventually, we may want to make this configurable.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message