spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject git commit: Merge pull request #169 from kayousterhout/mesos_fix
Date Thu, 14 Nov 2013 18:32:32 GMT
Updated Branches:
  refs/heads/branch-0.8 41dc566fe -> e7927ad3d


Merge pull request #169 from kayousterhout/mesos_fix

Don't ignore spark.cores.max when using Mesos Coarse mode

totalCoresAcquired is decremented but never incremented, causing Spark to effectively ignore
spark.cores.max in coarse grained Mesos mode.

(cherry picked from commit 1a4cfbea334c7b0dae287eab4c3131c8f4b8a992)
Signed-off-by: Reynold Xin <rxin@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/e7927ad3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/e7927ad3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/e7927ad3

Branch: refs/heads/branch-0.8
Commit: e7927ad3dba185e16ddf13f48d0b9f83c538de0c
Parents: 41dc566
Author: Reynold Xin <rxin@apache.org>
Authored: Thu Nov 14 10:32:11 2013 -0800
Committer: Reynold Xin <rxin@apache.org>
Committed: Thu Nov 14 10:32:27 2013 -0800

----------------------------------------------------------------------
 .../spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala | 1 +
 1 file changed, 1 insertion(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/e7927ad3/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
index 300fe69..cd521e0 100644
--- a/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
+++ b/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
@@ -181,6 +181,7 @@ private[spark] class CoarseMesosSchedulerBackend(
             !slaveIdsWithExecutors.contains(slaveId)) {
           // Launch an executor on the slave
           val cpusToUse = math.min(cpus, maxCores - totalCoresAcquired)
+          totalCoresAcquired += cpusToUse
           val taskId = newMesosTaskId()
           taskIdToSlaveId(taskId) = slaveId
           slaveIdsWithExecutors += slaveId


Mime
View raw message