spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andrewo...@apache.org
Subject spark git commit: [SPARK-7598] [DEPLOY] Add aliveWorkers metrics in Master
Date Thu, 14 May 2015 23:55:41 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.4 fceaffc49 -> 894214f9e


[SPARK-7598] [DEPLOY] Add aliveWorkers metrics in Master

In Spark Standalone setup, when some workers are DEAD, they will stay in master worker list
for a while.
master.workers metrics for master is only showing the total number of workers, we need to
monitor how many real ALIVE workers are there to ensure the cluster is healthy.

Author: Rex Xiong <pengx@microsoft.com>

Closes #6117 from twilightgod/add-aliveWorker-metrics and squashes the following commits:

6be69a5 [Rex Xiong] Fix comment for aliveWorkers metrics
a882f39 [Rex Xiong] Fix style for aliveWorkers metrics
38ce955 [Rex Xiong] Add aliveWorkers metrics in Master

(cherry picked from commit 93dbb3ad83fd60444a38c3dc87a2053c667123af)
Signed-off-by: Andrew Or <andrew@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/894214f9
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/894214f9
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/894214f9

Branch: refs/heads/branch-1.4
Commit: 894214f9eaf477c51c7b909f24e8be5c7d5343a5
Parents: fceaffc
Author: Rex Xiong <pengx@microsoft.com>
Authored: Thu May 14 16:55:31 2015 -0700
Committer: Andrew Or <andrew@databricks.com>
Committed: Thu May 14 16:55:37 2015 -0700

----------------------------------------------------------------------
 .../scala/org/apache/spark/deploy/master/MasterSource.scala     | 5 +++++
 1 file changed, 5 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/894214f9/core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala b/core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala
index 9c3f79f..66a9ff3 100644
--- a/core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala
+++ b/core/src/main/scala/org/apache/spark/deploy/master/MasterSource.scala
@@ -30,6 +30,11 @@ private[spark] class MasterSource(val master: Master) extends Source {
     override def getValue: Int = master.workers.size
   })
 
+  // Gauge for alive worker numbers in cluster
+  metricRegistry.register(MetricRegistry.name("aliveWorkers"), new Gauge[Int]{
+    override def getValue: Int = master.workers.filter(_.state == WorkerState.ALIVE).size
+  })
+
   // Gauge for application numbers in cluster
   metricRegistry.register(MetricRegistry.name("apps"), new Gauge[Int] {
     override def getValue: Int = master.apps.size


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message