hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "AbdulMateen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-18655) Apache hive 2.1.1 on Apache Spark 2.0
Date Thu, 08 Feb 2018 13:43:00 GMT
AbdulMateen created HIVE-18655:
----------------------------------

             Summary: Apache hive 2.1.1 on Apache Spark 2.0
                 Key: HIVE-18655
                 URL: https://issues.apache.org/jira/browse/HIVE-18655
             Project: Hive
          Issue Type: Bug
          Components: Hive, HiveServer2, Spark
    Affects Versions: 2.1.1
         Environment: apache hive  -2.1.1

apache spark - 2.0 - prebulit version (removed hive jars)

apache hadoop -2.8
            Reporter: AbdulMateen


Hi when connecting my beeline in hive it is not able to create spark client

{{select count(*) from student; Query ID = hadoop_20180208184224_f86b5aeb-f27b-4156-bd77-0aab54c0ec67
Total jobs = 1 Launching Job 1 out of 1 In order to change the average load for a reducer
(in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the
maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant
number of reducers: set mapreduce.job.reduces=<number>}}

{{FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
(state=08S01,code=1)}}{{}}
|Hi when connecting my beeline in hive it is not able to create spark client
{{select count(*) from student; Query ID = hadoop_20180208184224_f86b5aeb-f27b-4156-bd77-0aab54c0ec67
Total jobs = 1 Launching Job 1 out of 1 In order to change the average load for a reducer
(in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the
maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant
number of reducers: set mapreduce.job.reduces=<number> }}
Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed
to create spark client.)'
{{FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
(state=08S01,code=1) }}
Installed spark prebuilt 2.0 one in standalone cluster mode
My hive-site.xml -- placed in spark/conf too and removed the hive jars in hdfs path
{{<property> <name>spark.master</name> <value>yarn</value> <description>Spark
Master URL</description> </property> <property> <name>spark.eventLog.enabled</name>
<value>true</value> <description>Spark Event Log</description> </property>
<property> <name>spark.eventLog.dir</name> <value>hdfs://xx.xxx.xx.xx:9000/user/spark/eventLogging</value>
<description>Spark event log folder</description> </property> <property>
<name>spark.executor.memory</name> <value>512m</value> <description>Spark
executor memory</description> </property> <property> <name>spark.serializer</name>
<value>org.apache.spark.serializer.KryoSerializer</value> <description>Spark
serializer</description> </property> <property> <name>spark.yarn.jars</name>
<value>hdfs://xx.xxx.xx.xx:9000:/user/spark/spark-jars/*</value> </property>
<property> <name>spark.submit.deployMode</name> <value>cluster</value>
<description>Spark Master URL</description> </property }}
My yarn-site.xml
{{<property> <name>yarn.nodemanager.resource.memory-mb</name> <value>40960</value>
</property> <property> <name>yarn.scheduler.minimum-allocation-mb</name>
<value>2048</value> </property> <property> <name>yarn.scheduler.maximum-allocation-mb</name>
<value>8192</value> </property>}}|

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message