spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <>
Subject [jira] [Created] (SPARK-16630) Blacklist a node if executors won't launch on it.
Date Tue, 19 Jul 2016 19:49:20 GMT
Thomas Graves created SPARK-16630:

             Summary: Blacklist a node if executors won't launch on it.
                 Key: SPARK-16630
             Project: Spark
          Issue Type: Improvement
          Components: YARN
    Affects Versions: 1.6.2
            Reporter: Thomas Graves

On YARN, its possible that a node is messed or misconfigured such that a container won't launch
on it.  For instance if the Spark external shuffle handler didn't get loaded on it , maybe
its just some other transient error. 

It would be nice we could recognize this happening and stop trying to launch executors on
it since that could end up causing us to hit our max number of executor failures and then
kill the job.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message