hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits
Date Mon, 10 Sep 2018 00:36:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608607#comment-16608607
] 

Brock Noland commented on HIVE-20506:
-------------------------------------

[~stakiar] - 

{noformat}
-1	mvninstall	0m 49s	ql in the patch failed.
{noformat}

{noformat}
[ERROR] /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13685/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionManagerImpl.java:[99,31]
method initialize in class org.apache.hive.spark.client.SparkClientFactory cannot be applied
to given types;
  required: java.util.Map<java.lang.String,java.lang.String>
  found: java.util.Map<java.lang.String,java.lang.String>,org.apache.hadoop.hive.conf.HiveConf
  reason: actual and formal argument lists differ in length
{noformat}

Looks like it'd not picking up the change in {{spark-client}}, likely because mvn install
isn't being called from the root. I don't think think this is associated with the patch.

> HOS times out when cluster is full while Hive-on-MR waits
> ---------------------------------------------------------
>
>                 Key: HIVE-20506
>                 URL: https://issues.apache.org/jira/browse/HIVE-20506
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>            Priority: Major
>         Attachments: HIVE-20506-CDH5.14.2.patch, HIVE-20506.1.patch, HIVE-20506.2.patch,
HIVE-20506.3.patch, Screen Shot 2018-09-07 at 8.10.37 AM.png
>
>
> My understanding is as follows:
> Hive-on-MR when the cluster is full will wait for resources to be available before submitting
a job. This is because the hadoop jar command is the primary mechanism Hive uses to know if
a job is complete or failed.
>  
> Hive-on-Spark will timeout after {{SPARK_RPC_CLIENT_CONNECT_TIMEOUT}} because the RPC
client in the AppMaster doesn't connect back to the RPC Server in HS2. 
> This is a behavior difference it'd be great to close.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message