spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Or (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-8754) YarnClientSchedulerBackend doesn't stop gracefully in failure conditions
Date Thu, 02 Jul 2015 18:38:06 GMT

     [ https://issues.apache.org/jira/browse/SPARK-8754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Or updated SPARK-8754:
-----------------------------
    Target Version/s: 1.4.1, 1.5.0  (was: 1.5.0, 1.4.2)

> YarnClientSchedulerBackend doesn't stop gracefully in failure conditions
> ------------------------------------------------------------------------
>
>                 Key: SPARK-8754
>                 URL: https://issues.apache.org/jira/browse/SPARK-8754
>             Project: Spark
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.4.0
>            Reporter: Devaraj K
>            Priority: Minor
>             Fix For: 1.4.1, 1.5.0
>
>
> {code:xml}
> java.lang.NullPointerException
>         at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.stop(YarnClientSchedulerBackend.scala:151)
>         at org.apache.spark.scheduler.TaskSchedulerImpl.stop(TaskSchedulerImpl.scala:421)
>         at org.apache.spark.scheduler.DAGScheduler.stop(DAGScheduler.scala:1447)
>         at org.apache.spark.SparkContext.stop(SparkContext.scala:1651)
>         at org.apache.spark.SparkContext.<init>(SparkContext.scala:572)
>         at org.apache.spark.examples.SparkPi$.main(SparkPi.scala:28)
>         at org.apache.spark.examples.SparkPi.main(SparkPi.scala)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:621)
>         at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
>         at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
>         at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
>         at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {code}
> If the application has FINISHED/FAILED/KILLED or failed to launch application master,
monitorThread is not getting initialized but monitorThread.interrupt() is getting invoked
as part of stop() without any check and It is causing to throw NPE and also it is preventing
to stop the client.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message