hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <>
Subject [jira] [Commented] (HIVE-20506) HOS times out when cluster is full while Hive-on-MR waits
Date Mon, 10 Sep 2018 00:36:00 GMT


Brock Noland commented on HIVE-20506:

[~stakiar] - 

-1	mvninstall	0m 49s	ql in the patch failed.

[ERROR] /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13685/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/[99,31]
method initialize in class org.apache.hive.spark.client.SparkClientFactory cannot be applied
to given types;
  required: java.util.Map<java.lang.String,java.lang.String>
  found: java.util.Map<java.lang.String,java.lang.String>,org.apache.hadoop.hive.conf.HiveConf
  reason: actual and formal argument lists differ in length

Looks like it'd not picking up the change in {{spark-client}}, likely because mvn install
isn't being called from the root. I don't think think this is associated with the patch.

> HOS times out when cluster is full while Hive-on-MR waits
> ---------------------------------------------------------
>                 Key: HIVE-20506
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>            Priority: Major
>         Attachments: HIVE-20506-CDH5.14.2.patch, HIVE-20506.1.patch, HIVE-20506.2.patch,
HIVE-20506.3.patch, Screen Shot 2018-09-07 at 8.10.37 AM.png
> My understanding is as follows:
> Hive-on-MR when the cluster is full will wait for resources to be available before submitting
a job. This is because the hadoop jar command is the primary mechanism Hive uses to know if
a job is complete or failed.
> Hive-on-Spark will timeout after {{SPARK_RPC_CLIENT_CONNECT_TIMEOUT}} because the RPC
client in the AppMaster doesn't connect back to the RPC Server in HS2. 
> This is a behavior difference it'd be great to close.

This message was sent by Atlassian JIRA

View raw message