spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From caneGuy <...@git.apache.org>
Subject [GitHub] spark pull request #19942: [SPARK-22754][DEPLOY] Check whether spark.executo...
Date Wed, 13 Dec 2017 15:48:36 GMT
Github user caneGuy commented on a diff in the pull request:

    https://github.com/apache/spark/pull/19942#discussion_r156697267
  
    --- Diff: core/src/main/scala/org/apache/spark/SparkConf.scala ---
    @@ -564,6 +564,14 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging
with Seria
         val encryptionEnabled = get(NETWORK_ENCRYPTION_ENABLED) || get(SASL_ENCRYPTION_ENABLED)
         require(!encryptionEnabled || get(NETWORK_AUTH_ENABLED),
           s"${NETWORK_AUTH_ENABLED.key} must be enabled when enabling encryption.")
    +
    +    val executorTimeoutThreshold = getTimeAsSeconds("spark.network.timeout", "120s")
    +    val executorHeartbeatInterval = getTimeAsSeconds("spark.executor.heartbeatInterval",
"10s")
    +    // If spark.executor.heartbeatInterval bigger than spark.network.timeout,
    +    // it will almost always cause ExecutorLostFailure. See SPARK-22754.
    +    require(executorTimeoutThreshold > executorHeartbeatInterval, "The value of "
+
    +      s"'spark.network.timeout=${executorTimeoutThreshold}' must be no less than the
value of " +
    --- End diff --
    
    Agree with you @srowen Does new message make sense?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message