spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [SPARK-17087][MESOS] Documentation for Making Spark on Mesos honor port restrictions
Date Thu, 18 Aug 2016 11:19:24 GMT
Repository: spark
Updated Branches:
  refs/heads/master e82dbe600 -> b81421afb


[SPARK-17087][MESOS] Documentation for Making Spark on Mesos honor port restrictions

## What changes were proposed in this pull request?

- adds documentation for https://issues.apache.org/jira/browse/SPARK-11714

## How was this patch tested?
Doc no test needed.

Author: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com>

Closes #14667 from skonto/add_doc.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b81421af
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/b81421af
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/b81421af

Branch: refs/heads/master
Commit: b81421afb04959bb22b53653be0a09c1f1c5845f
Parents: e82dbe6
Author: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com>
Authored: Thu Aug 18 12:19:19 2016 +0100
Committer: Sean Owen <sowen@cloudera.com>
Committed: Thu Aug 18 12:19:19 2016 +0100

----------------------------------------------------------------------
 docs/running-on-mesos.md | 10 ++++++++++
 1 file changed, 10 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/b81421af/docs/running-on-mesos.md
----------------------------------------------------------------------
diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md
index a6ce34c..173961d 100644
--- a/docs/running-on-mesos.md
+++ b/docs/running-on-mesos.md
@@ -207,6 +207,16 @@ The scheduler will start executors round-robin on the offers Mesos
 gives it, but there are no spread guarantees, as Mesos does not
 provide such guarantees on the offer stream.
 
+In this mode spark executors will honor port allocation if such is
+provided from the user. Specifically if the user defines
+`spark.executor.port` or `spark.blockManager.port` in Spark configuration,
+the mesos scheduler will check the available offers for a valid port
+range containing the port numbers. If no such range is available it will
+not launch any task. If no restriction is imposed on port numbers by the
+user, ephemeral ports are used as usual. This port honouring implementation
+implies one task per host if the user defines a port. In the future network
+isolation shall be supported.
+
 The benefit of coarse-grained mode is much lower startup overhead, but
 at the cost of reserving Mesos resources for the complete duration of
 the application.  To configure your job to dynamically adjust to its


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message