spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From andrewo...@apache.org
Subject spark git commit: [SPARK-3674] YARN support in Spark EC2
Date Tue, 26 May 2015 22:01:48 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.4 51d98b0e9 -> d014a447a


[SPARK-3674] YARN support in Spark EC2

This corresponds to https://github.com/mesos/spark-ec2/pull/116 in the spark-ec2 repo. The
only changes required on the spark_ec2.py script is to open the RM port.

cc andrewor14

Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>

Closes #6376 from shivaram/spark-ec2-yarn and squashes the following commits:

961504a [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into
spark-ec2-yarn
152c94c [Shivaram Venkataraman] Open 8088 for YARN in EC2

(cherry picked from commit 2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a)
Signed-off-by: Andrew Or <andrew@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d014a447
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d014a447
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d014a447

Branch: refs/heads/branch-1.4
Commit: d014a447a361f65db8986e839e2f9a7a7a356f57
Parents: 51d98b0
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Authored: Tue May 26 15:01:27 2015 -0700
Committer: Andrew Or <andrew@databricks.com>
Committed: Tue May 26 15:01:44 2015 -0700

----------------------------------------------------------------------
 ec2/spark_ec2.py | 2 ++
 1 file changed, 2 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/d014a447/ec2/spark_ec2.py
----------------------------------------------------------------------
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py
index 2fab5c4..2154046 100755
--- a/ec2/spark_ec2.py
+++ b/ec2/spark_ec2.py
@@ -492,6 +492,8 @@ def launch_cluster(conn, opts, cluster_name):
         master_group.authorize('udp', 2049, 2049, authorized_address)
         master_group.authorize('tcp', 4242, 4242, authorized_address)
         master_group.authorize('udp', 4242, 4242, authorized_address)
+        # RM in YARN mode uses 8088
+        master_group.authorize('tcp', 8088, 8088, authorized_address)
         if opts.ganglia:
             master_group.authorize('tcp', 5080, 5080, authorized_address)
     if slave_group.rules == []:  # Group was just now created


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message