hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8780) insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
Date Fri, 07 Nov 2014 05:48:34 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201649#comment-14201649
] 

Chengxiang Li commented on HIVE-8780:
-------------------------------------

Hi, [~jxiang], SparkJobMonitor check job state in an loop until job finished/failed, and sleep
check interval time after checking, so it should be normal to get this stacktrace, SparkJobMonitor
should not block spark job execution as the job is submitted asynchronous before we call SparkJobMonitor::startMonitor.
could you test with a single qtest file and upload a full JVM stacktrace when it's long enough
to be sure job is hanging?

> insert1.q and ppd_join4.q hangs with hadoop-1 [Spark Branch]
> ------------------------------------------------------------
>
>                 Key: HIVE-8780
>                 URL: https://issues.apache.org/jira/browse/HIVE-8780
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Jimmy Xiang
>
> In working on HIVE-8758, found these tests hang at 
> {noformat}
>    java.lang.Thread.State: TIMED_WAITING (sleeping)
>         at java.lang.Thread.sleep(Native Method)
>         at org.apache.hadoop.hive.ql.exec.spark.status.SparkJobMonitor.startMoni
> tor(SparkJobMonitor.java:129)
>         at org.apache.hadoop.hive.ql.exec.spark.SparkTask.execute(SparkTask.java
> :111)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:161)
>         at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.ja
> va:85)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1644)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404)
>         at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1216)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1033)
>         at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:2
> 47)
>         at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:199)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
>         at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:345)
>         at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:832)
>         at org.apache.hadoop.hive.cli.TestSparkCliDriver.runTest(TestSparkCliDri
> ver.java:3706)
>         at org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_ppd_join4
> (TestSparkCliDriver.java:2790)
> {noformat}
> Both tests hang at the same place. There could be other hanging tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message