spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joshro...@apache.org
Subject spark git commit: [SPARK-4983] Insert waiting time before tagging EC2 instances
Date Fri, 06 Feb 2015 21:29:15 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.2 09da688b0 -> 36f70de83


[SPARK-4983] Insert waiting time before tagging EC2 instances

The boto API doesn't support tag EC2 instances in the same call that launches them.
We add a five-second wait so EC2 has enough time to propagate the information so that
the tagging can succeed.

Author: GenTang <gen.tang86@gmail.com>
Author: Gen TANG <gen.tang86@gmail.com>

Closes #3986 from GenTang/spark-4983 and squashes the following commits:

13e257d [Gen TANG] modification of comments
47f06755 [GenTang] print the information
ab7a931 [GenTang] solve the issus spark-4983 by inserting waiting time
3179737 [GenTang] Revert "handling exceptions about adding tags to ec2"
6a8b53b [GenTang] Revert "the improvement of exception handling"
13e97a6 [GenTang] Revert "typo"
63fd360 [GenTang] typo
692fc2b [GenTang] the improvement of exception handling
6adcf6d [GenTang] handling exceptions about adding tags to ec2

(cherry picked from commit 0f3a36071a44e986d97981032d5b192477b38bbd)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/36f70de8
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/36f70de8
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/36f70de8

Branch: refs/heads/branch-1.2
Commit: 36f70de8376317f7ab2ca0cbbe196001af8a3b44
Parents: 09da688
Author: GenTang <gen.tang86@gmail.com>
Authored: Fri Feb 6 13:27:34 2015 -0800
Committer: Josh Rosen <joshrosen@databricks.com>
Committed: Fri Feb 6 13:29:10 2015 -0800

----------------------------------------------------------------------
 ec2/spark_ec2.py | 3 +++
 1 file changed, 3 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/36f70de8/ec2/spark_ec2.py
----------------------------------------------------------------------
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py
index a137f89..aaddf61 100755
--- a/ec2/spark_ec2.py
+++ b/ec2/spark_ec2.py
@@ -480,6 +480,9 @@ def launch_cluster(conn, opts, cluster_name):
         master_nodes = master_res.instances
         print "Launched master in %s, regid = %s" % (zone, master_res.id)
 
+    # This wait time corresponds to SPARK-4983
+    print "Waiting for AWS to propagate instance metadata..."
+    time.sleep(5)
     # Give the instances descriptive names
     for master in master_nodes:
         master.add_tag(


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message