hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mich Talebzadeh" <m...@peridale.co.uk>
Subject RE: Hive on Spark - Error: Child process exited before connecting back
Date Tue, 15 Dec 2015 22:33:03 GMT
Hi,

 

The only version that I have managed to run Hive using Spark engine is Spark 1.3.1 on Hive
1.2.1

 

Can you confirm the version of Spark you are running?

 

FYI, Spark 1.5.2 will not work with Hive.

 

HTH

 

Mich Talebzadeh

 

Sybase ASE 15 Gold Medal Award 2008

A Winning Strategy: Running the most Critical Financial Data on ASE 15

http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf

Author of the books "A Practitioner’s Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7.


co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4

Publications due shortly:

Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8

Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly

 

http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/> 

 

NOTE: The information in this email is proprietary and confidential. This message is for the
designated recipient only, if you are not the intended recipient, you should destroy it immediately.
Any information in this message shall not be understood as given or endorsed by Peridale Technology
Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility
of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd,
its subsidiaries nor their employees accept any responsibility.

 

From: Ophir Etzion [mailto:ophir@foursquare.com] 
Sent: 15 December 2015 22:27
To: user@spark.apache.org; user@hive.apache.org
Subject: Hive on Spark - Error: Child process exited before connecting back

 

Hi,

 

when trying to do Hive on Spark on CDH5.4.3 I get the following error when trying to run a
simple query using spark.

I've tried setting everything written here (https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started)
as well as what the cdh recommends.

any one encountered this as well? (searching for it didn't help much)

the error:

ERROR : Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed
to create spark client.)'

org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client.

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:57)

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:114)

            at org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:120)

            at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:97)

            at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)

            at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)

            at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640)

            at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399)

            at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183)

            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)

            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)

            at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144)

            at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69)

            at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196)

            at java.security.AccessController.doPrivileged(Native Method)

            at javax.security.auth.Subject.doAs(Subject.java:415)

            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)

            at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)

            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

            at java.util.concurrent.FutureTask.run(FutureTask.java:262)

            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

            at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
Cancel client '2b2d7314-e0cc-4933-82a1-992a3299d109'. Error: Child process exited before connecting
back

            at com.google.common.base.Throwables.propagate(Throwables.java:156)

            at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:109)

            at org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:80)

            at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.<init>(RemoteHiveSparkClient.java:91)

            at org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:65)

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:55)

            ... 22 more

Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Cancel client
'2b2d7314-e0cc-4933-82a1-992a3299d109'. Error: Child process exited before connecting back

            at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)

            at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:99)

            ... 26 more

Caused by: java.lang.RuntimeException: Cancel client '2b2d7314-e0cc-4933-82a1-992a3299d109'.
Error: Child process exited before connecting back

            at org.apache.hive.spark.client.rpc.RpcServer.cancelClient(RpcServer.java:179)

            at org.apache.hive.spark.client.SparkClientImpl$3.run(SparkClientImpl.java:427)

            ... 1 more

 

ERROR : Failed to execute spark task, with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(Failed
to create spark client.)'

org.apache.hadoop.hive.ql.metadata.HiveException: Failed to create spark client.

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:57)

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl.getSession(SparkSessionManagerImpl.java:114)

            at org.apache.hadoop.hive.ql.exec.spark.SparkUtilities.getSparkSession(SparkUtilities.java:120)

            at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java:97)

            at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)

            at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88)

            at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1640)

            at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1399)

            at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1183)

            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)

            at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1044)

            at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:144)

            at org.apache.hive.service.cli.operation.SQLOperation.access$100(SQLOperation.java:69)

            at org.apache.hive.service.cli.operation.SQLOperation$1$1.run(SQLOperation.java:196)

            at java.security.AccessController.doPrivileged(Native Method)

            at javax.security.auth.Subject.doAs(Subject.java:415)

            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)

            at org.apache.hive.service.cli.operation.SQLOperation$1.run(SQLOperation.java:208)

            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)

            at java.util.concurrent.FutureTask.run(FutureTask.java:262)

            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)

            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

            at java.lang.Thread.run(Thread.java:745)

Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException:
Cancel client '2b2d7314-e0cc-4933-82a1-992a3299d109'. Error: Child process exited before connecting
back

            at com.google.common.base.Throwables.propagate(Throwables.java:156)

            at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:109)

            at org.apache.hive.spark.client.SparkClientFactory.createClient(SparkClientFactory.java:80)

            at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient.<init>(RemoteHiveSparkClient.java:91)

            at org.apache.hadoop.hive.ql.exec.spark.HiveSparkClientFactory.createHiveSparkClient(HiveSparkClientFactory.java:65)

            at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.open(SparkSessionImpl.java:55)

            ... 22 more

Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Cancel client
'2b2d7314-e0cc-4933-82a1-992a3299d109'. Error: Child process exited before connecting back

            at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)

            at org.apache.hive.spark.client.SparkClientImpl.<init>(SparkClientImpl.java:99)

            ... 26 more

Caused by: java.lang.RuntimeException: Cancel client '2b2d7314-e0cc-4933-82a1-992a3299d109'.
Error: Child process exited before connecting back

            at org.apache.hive.spark.client.rpc.RpcServer.cancelClient(RpcServer.java:179)

            at org.apache.hive.spark.client.SparkClientImpl$3.run(SparkClientImpl.java:427)

            ... 1 more

Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
(state=08S01,code=1)

 


Mime
View raw message