spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tgravescs <...@git.apache.org>
Subject [GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...
Date Tue, 17 Apr 2018 14:13:23 GMT
Github user tgravescs commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21068#discussion_r181882445
  
    --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
---
    @@ -328,4 +328,26 @@ package object config {
         CACHED_FILES_TYPES,
         CACHED_CONF_ARCHIVE)
     
    +  /* YARN allocator-level blacklisting related config entries. */
    +  private[spark] val YARN_ALLOCATION_BLACKLIST_ENABLED =
    +    ConfigBuilder("spark.yarn.allocation.blacklist.enabled")
    +      .booleanConf
    +      .createOptional
    +
    +  private[spark] val YARN_BLACKLIST_SIZE_LIMIT =
    +    ConfigBuilder("spark.yarn.blacklist.size.limit")
    +      .doc("Limit for blacklisted nodes. This is the maximum for the number of blacklisted
nodes " +
    +        "(including task and stage blacklisted nodes) sent to YARN.")
    +      .intConf
    +      .createOptional
    +
    +  private[spark] val YARN_BLACKLIST_SIZE_DEFAULT_WEIGHT =
    +  ConfigBuilder("spark.yarn.blacklist.size.default.weight")
    +    .doc("If blacklist size limit is not specified then the default limit will be the
number of " +
    --- End diff --
    
    perhaps rename something like spark.yarn.blacklist.maxNodeBlacklistRatio . (note we are
talking about using Ratio in another config here: https://github.com/apache/spark/pull/19881)


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message