spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicholas Chammas (JIRA)" <>
Subject [jira] [Commented] (SPARK-6900) spark ec2 script enters infinite loop when run-instance fails
Date Fri, 17 Apr 2015 19:50:58 GMT


Nicholas Chammas commented on SPARK-6900:

Hi [], I would not call this a major bug (as the issue is currently labeled).

When the situation you are describing happens, spark-ec2 prints out notices about the SSH
failures. This should give the user enough information to debug the problem. Don't you agree?

If the problem is indeed that an instance was terminated before it came up, the user can simply
cancel the launch and restart it with the {{--resume}} flag. Is that not a satisfactory solution?

Having spark-ec2 automatically handle instances that terminate prematurely on launch would
be a nice to have, but it seems like such a niche feature for a rare occurrence that I do
not think it's worth the investment. 

If this all makes sense to you, I suggest closing this issue.

> spark ec2 script enters infinite loop when run-instance fails
> -------------------------------------------------------------
>                 Key: SPARK-6900
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: EC2
>    Affects Versions: 1.3.0
>            Reporter: Guodong Wang
> I am using spark-ec2 scripts to launch spark cluters in AWS.
> Recently, in our AWS region,  there were some tech issues about AWS EC2 service. 
> When spark-ec2 send the run-instance requests to EC2, not all the requested instances
were launched. Some instance was terminated by AWS-EC2 service  before it was up.
> But spark-ec2 script would wait for all the instances to enter 'ssh-ready' status. So,
the script enters the infinite loop. Because the terminated instances would never be 'ssh-ready'.
> In my opinion, it should be OK if some of the slave instances were terminated. As long
as the master node is running, the terminated slaves should be filtered and the cluster should
be setup.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message