spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sro...@apache.org
Subject spark git commit: [EC2] [SPARK-6188] Instance types can be mislabeled when re-starting cluster with default arguments
Date Mon, 09 Mar 2015 14:16:13 GMT
Repository: spark
Updated Branches:
  refs/heads/master 55b1b32dc -> f7c799204


[EC2] [SPARK-6188] Instance types can be mislabeled when re-starting cluster with default
arguments

As described in https://issues.apache.org/jira/browse/SPARK-6188 and discovered in https://issues.apache.org/jira/browse/SPARK-5838.

When re-starting a cluster, if the user does not provide the instance types, which is the
recommended behavior in the docs currently, the instance will be assigned the default type
m1.large. This then affects the setup of the machines.

This solves this by getting the instance types from the existing instances, and overwriting
the default options.

EDIT: Further clarification of the issue:

In short, while the instances themselves are the same as launched, their setup is done assuming
the default instance type, m1.large.

This means that the machines are assumed to have 2 disks, and that leads to problems that
are described in in issue [5838](https://issues.apache.org/jira/browse/SPARK-5838), where
machines that have one disk end up having shuffle spills in the in the small (8GB) snapshot
partitions that quickly fills up and results in failing jobs due to "No space left on device"
errors.

Other instance specific settings that are set in the spark_ec2.py script are likely to be
wrong as well.

Author: Theodore Vasiloudis <thvasilo@users.noreply.github.com>
Author: Theodore Vasiloudis <tvas@sics.se>

Closes #4916 from thvasilo/SPARK-6188]-Instance-types-can-be-mislabeled-when-re-starting-cluster-with-default-arguments
and squashes the following commits:

6705b98 [Theodore Vasiloudis] Added comment to clarify setting master instance type to the
empty string.
a3d29fe [Theodore Vasiloudis] More trailing whitespace
7b32429 [Theodore Vasiloudis] Removed trailing whitespace
3ebd52a [Theodore Vasiloudis] Make sure that the instance type is correct when relaunching
a cluster.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f7c79920
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f7c79920
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f7c79920

Branch: refs/heads/master
Commit: f7c799204358bcc38c5972a29e5994b78b25b515
Parents: 55b1b32
Author: Theodore Vasiloudis <thvasilo@users.noreply.github.com>
Authored: Mon Mar 9 14:16:07 2015 +0000
Committer: Sean Owen <sowen@cloudera.com>
Committed: Mon Mar 9 14:16:07 2015 +0000

----------------------------------------------------------------------
 ec2/spark_ec2.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/f7c79920/ec2/spark_ec2.py
----------------------------------------------------------------------
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py
index 5e636dd..b50b381 100755
--- a/ec2/spark_ec2.py
+++ b/ec2/spark_ec2.py
@@ -1307,6 +1307,17 @@ def real_main():
             cluster_instances=(master_nodes + slave_nodes),
             cluster_state='ssh-ready'
         )
+
+        # Determine types of running instances
+        existing_master_type = master_nodes[0].instance_type
+        existing_slave_type = slave_nodes[0].instance_type
+        # Setting opts.master_instance_type to the empty string indicates we
+        # have the same instance type for the master and the slaves
+        if existing_master_type == existing_slave_type:
+            existing_master_type = ""
+        opts.master_instance_type = existing_master_type
+        opts.instance_type = existing_slave_type
+
         setup_cluster(conn, master_nodes, slave_nodes, opts, False)
 
     else:


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message