hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-11969) start Tez session in background when starting CLI
Date Wed, 30 Sep 2015 23:30:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-11969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14939055#comment-14939055
] 

Siddharth Seth commented on HIVE-11969:
---------------------------------------

With the patch - prewarm is moving into the async initialization. However I don't think that
completely defeats the purpose of prewarm. Containers will still be started in the background
if cluster capacity is available. The user query can get delayed though while it waits for
the prewarm to complete.

IMHO, adding a log to the CLI indicating that a query is waiting for cluster resources is
better than blocking the entire CLI from starting up. Also, as always, a configuration to
fallback to current behaviour - which in this case would be disabled by default. Pre-warm
as it exists today, would run with this set appropriately.

> start Tez session in background when starting CLI
> -------------------------------------------------
>
>                 Key: HIVE-11969
>                 URL: https://issues.apache.org/jira/browse/HIVE-11969
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-11969.01.patch, HIVE-11969.patch
>
>
> Tez session spins up AM, which can cause delays, esp. if the cluster is very busy.
> This can be done in background, so the AM might get started while the user is running
local commands and doing other things.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message