hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaomeng Huang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14369) submit a task with hive on spark to other yarn cluster failed
Date Fri, 29 Jul 2016 09:08:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399006#comment-15399006
] 

Xiaomeng Huang commented on HIVE-14369:
---------------------------------------

Hi [~ruili] 
It can't work, too. I think I should set both hivecluster and sparkcluster in hdfs-site.xml
of sparkcluster.

> submit a task with hive on spark to other yarn cluster failed
> -------------------------------------------------------------
>
>                 Key: HIVE-14369
>                 URL: https://issues.apache.org/jira/browse/HIVE-14369
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Xiaomeng Huang
>
> In our environment, we have two hadoop clusters with HA,  named hivecluster and sparkcluster.
> hivecluster is a HA hadoop cluster for hive, which has large hard disk.
> sparkcluster is a HA hadoop cluster for spark, which has large memory.
> e.g. below is a hdfs-site.xml of hivecluster:
> {code}
> <property>
>         <name>dfs.ha.namenodes.hivecluster</name>
>         <value>nn1,nn2</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.hivecluster.nn1</name>
>         <value>10.17.21.32:9000</value>
> </property>
> <property>
>         <name>dfs.namenode.rpc-address.hivecluster.nn2</name>
>         <value>10.17.21.77:9000</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.hivecluster.nn1</name>
>         <value>10.17.21.32:50070</value>
> </property>
> <property>
>         <name>dfs.namenode.http-address.hivecluster.nn2</name>
>         <value>10.17.21.77:50070</value>
> </property>
> {code}
> Firstly, I created a hive table located as hdfs://hivecluster/hive/warehouse/xxx
> If I use hive on mr, it will run successfully.
> But if I use hive on spark to submit a task to yarn cluster of sparkcluster, it says:
> {code}
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask
> yarn日志显示:
> Diagnostics: java.lang.IllegalArgumentException: java.net.UnknownHostException: hivecluster
> Failing this attempt. Failing the application.
> {code}
> I didn't set host of hivecluster into hdfs-site.xml of sparkcluster



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message