Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5572010FED for ; Thu, 22 Jan 2015 13:46:35 +0000 (UTC) Received: (qmail 54354 invoked by uid 500); 22 Jan 2015 13:46:35 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 54279 invoked by uid 500); 22 Jan 2015 13:46:34 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 54267 invoked by uid 500); 22 Jan 2015 13:46:34 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 54264 invoked by uid 99); 22 Jan 2015 13:46:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 22 Jan 2015 13:46:34 +0000 Date: Thu, 22 Jan 2015 13:46:34 +0000 (UTC) From: "Hive QA (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-9370) SparkJobMonitor timeout as sortByKey would launch extra Spark job before original job get submitted [Spark Branch] MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-9370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14287425#comment-14287425 ] Hive QA commented on HIVE-9370: ------------------------------- {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12693867/HIVE-9370.1-spark.patch {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 7351 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_vectorized_timestamp_funcs org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_optimize_nullscan {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/669/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/669/console Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-669/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12693867 - PreCommit-HIVE-SPARK-Build > SparkJobMonitor timeout as sortByKey would launch extra Spark job before original job get submitted [Spark Branch] > ------------------------------------------------------------------------------------------------------------------ > > Key: HIVE-9370 > URL: https://issues.apache.org/jira/browse/HIVE-9370 > Project: Hive > Issue Type: Sub-task > Components: Spark > Reporter: yuyun.chen > Assignee: Chengxiang Li > Attachments: HIVE-9370.1-spark.patch > > > enable hive on spark and run BigBench Query 8 then got the following exception: > 2015-01-14 11:43:46,057 INFO [main]: impl.RemoteSparkJobStatus (RemoteSparkJobStatus.java:getSparkJobInfo(143)) - Job hasn't been submitted after 30s. Aborting it. > 2015-01-14 11:43:46,061 INFO [main]: impl.RemoteSparkJobStatus (RemoteSparkJobStatus.java:getSparkJobInfo(143)) - Job hasn't been submitted after 30s. Aborting it. > 2015-01-14 11:43:46,061 ERROR [main]: status.SparkJobMonitor (SessionState.java:printError(839)) - Status: Failed > 2015-01-14 11:43:46,062 INFO [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - 15/01/14 11:43:46 INFO RemoteDriver: Failed to run job 0a9a7782-0e0b-4561-8468-959a6d8df0a3 > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - java.lang.InterruptedException > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.lang.Object.wait(Native Method) > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.lang.Object.wait(Object.java:503) > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.scheduler.JobWaiter.awaitResult(JobWaiter.scala:73) > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:514) > 2015-01-14 11:43:46,071 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.SparkContext.runJob(SparkContext.scala:1282) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.SparkContext.runJob(SparkContext.scala:1300) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.SparkContext.runJob(SparkContext.scala:1314) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.SparkContext.runJob(SparkContext.scala:1328) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.rdd.RDD.collect(RDD.scala:780) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.RangePartitioner$.sketch(Partitioner.scala:262) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.RangePartitioner.(Partitioner.scala:124) > 2015-01-14 11:43:46,072 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.rdd.OrderedRDDFunctions.sortByKey(OrderedRDDFunctions.scala:63) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.api.java.JavaPairRDD.sortByKey(JavaPairRDD.scala:894) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.spark.api.java.JavaPairRDD.sortByKey(JavaPairRDD.scala:864) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hadoop.hive.ql.exec.spark.SortByShuffler.shuffle(SortByShuffler.java:48) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hadoop.hive.ql.exec.spark.ShuffleTran.transform(ShuffleTran.java:45) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hadoop.hive.ql.exec.spark.SparkPlan.generateGraph(SparkPlan.java:69) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient$JobStatusJob.call(RemoteHiveSparkClient.java:223) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:298) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:269) > 2015-01-14 11:43:46,073 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.util.concurrent.FutureTask.run(FutureTask.java:262) > 2015-01-14 11:43:46,074 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > 2015-01-14 11:43:46,074 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > 2015-01-14 11:43:46,074 INFO [stderr-redir-1]: client.SparkClientImpl (SparkClientImpl.java:run(436)) - at java.lang.Thread.run(Thread.java:745) > 2015-01-14 11:43:46,077 WARN [RPC-Handler-3]: client.SparkClientImpl (SparkClientImpl.java:handle(407)) - Received result for unknown job 0a9a7782-0e0b-4561-8468-959a6d8df0a3 > 2015-01-14 11:43:46,091 ERROR [main]: ql.Driver (SessionState.java:printError(839)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.spark.SparkTask -- This message was sent by Atlassian JIRA (v6.3.4#6332)