spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From r...@apache.org
Subject spark git commit: [SPARK-13414][MESOS] Allow multiple dispatchers to be launched.
Date Sat, 20 Feb 2016 20:58:54 GMT
Repository: spark
Updated Branches:
  refs/heads/master 6ce7c481d -> a4a081d1d


[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.

## What changes were proposed in this pull request?

Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be
part of different roles and used for multi-tenancy.

To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance
number when starting the dispatcher daemon.

## How was the this patch tested?

Manual testing

Author: Timothy Chen <tnachen@gmail.com>

Closes #11281 from tnachen/multiple_cluster_dispatchers.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a4a081d1
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/a4a081d1
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/a4a081d1

Branch: refs/heads/master
Commit: a4a081d1df043cce6db0284ef552e5174ebb0d02
Parents: 6ce7c48
Author: Timothy Chen <tnachen@gmail.com>
Authored: Sat Feb 20 12:58:47 2016 -0800
Committer: Reynold Xin <rxin@databricks.com>
Committed: Sat Feb 20 12:58:47 2016 -0800

----------------------------------------------------------------------
 sbin/start-mesos-dispatcher.sh | 5 ++++-
 sbin/stop-mesos-dispatcher.sh  | 7 ++++++-
 2 files changed, 10 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/a4a081d1/sbin/start-mesos-dispatcher.sh
----------------------------------------------------------------------
diff --git a/sbin/start-mesos-dispatcher.sh b/sbin/start-mesos-dispatcher.sh
index 4777e16..06a966d 100755
--- a/sbin/start-mesos-dispatcher.sh
+++ b/sbin/start-mesos-dispatcher.sh
@@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
   SPARK_MESOS_DISPATCHER_HOST=`hostname`
 fi
 
+if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
+  SPARK_MESOS_DISPATCHER_NUM=1
+fi
 
-"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher
1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
+"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher
$SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT
"$@"

http://git-wip-us.apache.org/repos/asf/spark/blob/a4a081d1/sbin/stop-mesos-dispatcher.sh
----------------------------------------------------------------------
diff --git a/sbin/stop-mesos-dispatcher.sh b/sbin/stop-mesos-dispatcher.sh
index 5c0b4e0..b13e018 100755
--- a/sbin/stop-mesos-dispatcher.sh
+++ b/sbin/stop-mesos-dispatcher.sh
@@ -24,5 +24,10 @@ fi
 
 . "${SPARK_HOME}/sbin/spark-config.sh"
 
-"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher
1
+if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
+  SPARK_MESOS_DISPATCHER_NUM=1
+fi
+
+"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher
\
+    $SPARK_MESOS_DISPATCHER_NUM
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message