hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-8780) insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
Date Wed, 12 Nov 2014 05:49:34 GMT

     [ https://issues.apache.org/jira/browse/HIVE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chengxiang Li updated HIVE-8780:
--------------------------------
    Attachment: 8780.1-spark.patch

Spark job with empty source data would not be submmited to spark cluster actually, so no JobEnd/JobStart
event are posted to Spark listener bus. As Hive monitor spark job state by spark listener,
it would never get any job state, we use JavaFutureAction to fetch job state in this patch.

> insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
> ------------------------------------------------------------
>
>                 Key: HIVE-8780
>                 URL: https://issues.apache.org/jira/browse/HIVE-8780
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Jimmy Xiang
>         Attachments: HIVE-8780.1-spark.patch, insert1.q-spark.png, insert1.q.jstack,
itests.patch
>
>
> In working on HIVE-8758, found these tests hang at 
> {noformat}
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>         at java.lang.Thread.sleep(Native Method)
>         at org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor.startMoni
> tor(SparkJobMonitor.java:129)
>         at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java
> :111)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:161)
>         at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
> va:85)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404)
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1216)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033)
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:2
> 47)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:345)
>         at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:832)
>         at org.apache.hadoop.hive.cli.TestSparkCliDriver.runTest(TestSparkCliDri
> ver.java:3706)
>         at org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ppd_join4
> (TestSparkCliDriver.java:2790)
> {noformat}
> Both tests hang at the same place. There could be other hanging tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message