hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rui Li (JIRA)" <>
Subject [jira] [Commented] (HIVE-12951) Reduce Spark executor prewarm timeout to 5s
Date Tue, 25 Oct 2016 12:01:58 GMT


Rui Li commented on HIVE-12951:

Hi [~richard_xin], pre-warm is intended to wait until some containers are launched before
we decide the parallelism/start the job. In general, it helps us to have a higher parallelism
and better task scheduling. However it doesn't remove the start-up overhead for the 1st query
- anyway we have to wait for the containers to be launched to run the query.
[~xuefuz] please add if I missed anything.

> Reduce Spark executor prewarm timeout to 5s
> -------------------------------------------
>                 Key: HIVE-12951
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.2.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>             Fix For: 2.1.0
>         Attachments: HIVE-12951.patch, HIVE-12951.patch
> Currently it's set to 30s, which tends to be longer than needed. Reduce it to 5s, only
considering jvm startup time. (Eventually, we may want to make this configurable.)

This message was sent by Atlassian JIRA

View raw message