spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From van...@apache.org
Subject spark git commit: [SPARK-13642][YARN] Changed the default application exit state to failed for yarn cluster mode
Date Tue, 15 Mar 2016 17:20:55 GMT
Repository: spark
Updated Branches:
  refs/heads/master 5e6f2f456 -> d89c71417


[SPARK-13642][YARN] Changed the default application exit state to failed for yarn cluster
mode

## What changes were proposed in this pull request?

Changing the default exit state to `failed` for any application running on yarn cluster mode.

## How was this patch tested?

Unit test is done locally.

CC tgravescs and vanzin .

Author: jerryshao <sshao@hortonworks.com>

Closes #11693 from jerryshao/SPARK-13642.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d89c7141
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d89c7141
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d89c7141

Branch: refs/heads/master
Commit: d89c71417c384e1a2820bd80b67c26e11405aebc
Parents: 5e6f2f4
Author: jerryshao <sshao@hortonworks.com>
Authored: Tue Mar 15 10:20:38 2016 -0700
Committer: Marcelo Vanzin <vanzin@cloudera.com>
Committed: Tue Mar 15 10:20:38 2016 -0700

----------------------------------------------------------------------
 .../apache/spark/deploy/yarn/ApplicationMaster.scala | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/d89c7141/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
----------------------------------------------------------------------
diff --git a/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala b/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
index 7d7bf88..cd179cf 100644
--- a/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
+++ b/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
@@ -152,13 +152,13 @@ private[spark] class ApplicationMaster(
         val isLastAttempt = client.getAttemptId().getAttemptId() >= maxAppAttempts
 
         if (!finished) {
-          // This happens when the user application calls System.exit(). We have the choice
-          // of either failing or succeeding at this point. We report success to avoid
-          // retrying applications that have succeeded (System.exit(0)), which means that
-          // applications that explicitly exit with a non-zero status will also show up as
-          // succeeded in the RM UI.
+          // The default state of ApplicationMaster is failed if it is invoked by shut down
hook.
+          // This behavior is different compared to 1.x version.
+          // If user application is exited ahead of time by calling System.exit(N), here
mark
+          // this application as failed with EXIT_EARLY. For a good shutdown, user shouldn't
call
+          // System.exit(0) to terminate the application.
           finish(finalStatus,
-            ApplicationMaster.EXIT_SUCCESS,
+            ApplicationMaster.EXIT_EARLY,
             "Shutdown hook called before final status was reported.")
         }
 
@@ -209,7 +209,7 @@ private[spark] class ApplicationMaster(
    */
   final def getDefaultFinalStatus(): FinalApplicationStatus = {
     if (isClusterMode) {
-      FinalApplicationStatus.SUCCEEDED
+      FinalApplicationStatus.FAILED
     } else {
       FinalApplicationStatus.UNDEFINED
     }
@@ -653,6 +653,7 @@ object ApplicationMaster extends Logging {
   private val EXIT_SC_NOT_INITED = 13
   private val EXIT_SECURITY = 14
   private val EXIT_EXCEPTION_USER_CLASS = 15
+  private val EXIT_EARLY = 16
 
   private var master: ApplicationMaster = _
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message