hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12650) Spark-submit is killed when Hive times out. Killing spark-submit doesn't cancel AM request. When AM is finally launched, it tries to connect back to Hive and gets refused.
Date Mon, 21 Mar 2016 21:31:25 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15205200#comment-15205200
] 

Xuefu Zhang commented on HIVE-12650:
------------------------------------

Since Hive cannot really differentiate the scenarios, I'm not sure if there is anything we
can do better except for better error message and documentation. RPC timeout is necessary
due to network.

On a side note, Yarn queues are more appropriate for solving the starvation problem. The problem
here seems more like an uncommon scenario.

> Spark-submit is killed when Hive times out. Killing spark-submit doesn't cancel AM request.
When AM is finally launched, it tries to connect back to Hive and gets refused.
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-12650
>                 URL: https://issues.apache.org/jira/browse/HIVE-12650
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.1.1, 1.2.1
>            Reporter: JoneZhang
>            Assignee: Xuefu Zhang
>
> I think hive.spark.client.server.connect.timeout should be set greater than spark.yarn.am.waitTime.
The default value for 
> spark.yarn.am.waitTime is 100s, and the default value for hive.spark.client.server.connect.timeout
is 90s, which is not good. We can increase it to a larger value such as 120s.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message