spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Graves (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-9823) Spark application hangs when using dynamic allocations and executor fails connect to shuffle service
Date Tue, 11 Aug 2015 15:54:45 GMT
Thomas Graves created SPARK-9823:
------------------------------------

             Summary: Spark application hangs when using dynamic allocations and executor
fails connect to shuffle service
                 Key: SPARK-9823
                 URL: https://issues.apache.org/jira/browse/SPARK-9823
             Project: Spark
          Issue Type: Bug
          Components: YARN
    Affects Versions: 1.5.0
            Reporter: Thomas Graves
            Priority: Critical


I was trying out spark 1.5 with the dynamic allocations turned on in YARN.  If the executor
fails to connect to the external shuffle service then the application just hangs there.  The
SparkContext didn't seem to be doing anything or requesting any new executors to replace it.
 

In this case the shuffle service had security on but I forgot to turn it on in the application.

====
This is somewhat a tangent but the reason I tried spark 1.5 is with spark1.4.1 I also found
the application hung with no executors so it must not have been requesting them properly.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message