spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tgra...@apache.org
Subject git commit: SPARK-2400 : fix spark.yarn.max.executor.failures explaination
Date Tue, 08 Jul 2014 18:55:50 GMT
Repository: spark
Updated Branches:
  refs/heads/master c8a2313cd -> b520b6453


SPARK-2400 : fix spark.yarn.max.executor.failures explaination

According to
```scala
  private val maxNumExecutorFailures = sparkConf.getInt("spark.yarn.max.executor.failures",
    sparkConf.getInt("spark.yarn.max.worker.failures", math.max(args.numExecutors * 2, 3)))
```
default value should be numExecutors * 2, with minimum of 3,  and it's same to the config
`spark.yarn.max.worker.failures`

Author: CrazyJvm <crazyjvm@gmail.com>

Closes #1282 from CrazyJvm/yarn-doc and squashes the following commits:

1a5f25b [CrazyJvm] remove deprecated config
c438aec [CrazyJvm] fix style
86effa6 [CrazyJvm] change expression
211f130 [CrazyJvm] fix html tag
2900d23 [CrazyJvm] fix style
a4b2e27 [CrazyJvm] fix configuration spark.yarn.max.executor.failures


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b520b645
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b520b645
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b520b645

Branch: refs/heads/master
Commit: b520b6453ed76926108e0bdd56114d16e1d86850
Parents: c8a2313
Author: CrazyJvm <crazyjvm@gmail.com>
Authored: Tue Jul 8 13:55:42 2014 -0500
Committer: Thomas Graves <tgraves@apache.org>
Committed: Tue Jul 8 13:55:42 2014 -0500

----------------------------------------------------------------------
 docs/running-on-yarn.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b520b645/docs/running-on-yarn.md
----------------------------------------------------------------------
diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md
index 5d8d603..0362f5a 100644
--- a/docs/running-on-yarn.md
+++ b/docs/running-on-yarn.md
@@ -55,7 +55,7 @@ Most of the configs are the same for Spark on YARN as for other deployment
modes
 </tr>
 <tr>
   <td><code>spark.yarn.max.executor.failures</code></td>
-  <td>2*numExecutors</td>
+  <td>numExecutors * 2, with minimum of 3</td>
   <td>
     The maximum number of executor failures before failing the application.
   </td>


Mime
View raw message