spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andrewo...@apache.org
Subject spark git commit: SPARK-4300 [CORE] Race condition during SparkWorker shutdown
Date Thu, 26 Feb 2015 22:08:58 GMT
Repository: spark
Updated Branches:
  refs/heads/master 5f3238b3b -> 3fb53c029


SPARK-4300 [CORE] Race condition during SparkWorker shutdown

Close appender saving stdout/stderr before destroying process to avoid exception on reading
closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)

CC tdas since I think you wrote this method.

Author: Sean Owen <sowen@cloudera.com>

Closes #4787 from srowen/SPARK-4300 and squashes the following commits:

e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying process to avoid
exception on reading closed input stream


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/3fb53c02
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/3fb53c02
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/3fb53c02

Branch: refs/heads/master
Commit: 3fb53c0298761ba227890525ae79ce4ec6300deb
Parents: 5f3238b
Author: Sean Owen <sowen@cloudera.com>
Authored: Thu Feb 26 14:08:56 2015 -0800
Committer: Andrew Or <andrew@databricks.com>
Committed: Thu Feb 26 14:08:56 2015 -0800

----------------------------------------------------------------------
 .../scala/org/apache/spark/deploy/worker/ExecutorRunner.scala     | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/3fb53c02/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
index 6653aca..066d46c 100644
--- a/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala
@@ -85,14 +85,13 @@ private[spark] class ExecutorRunner(
     var exitCode: Option[Int] = None
     if (process != null) {
       logInfo("Killing process!")
-      process.destroy()
-      process.waitFor()
       if (stdoutAppender != null) {
         stdoutAppender.stop()
       }
       if (stderrAppender != null) {
         stderrAppender.stop()
       }
+      process.destroy()
       exitCode = Some(process.waitFor())
     }
     worker ! ExecutorStateChanged(appId, execId, state, message, exitCode)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message