spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-1755) Spark-submit --name does not resolve to application name on YARN
Date Sat, 10 May 2014 22:13:14 GMT

     [ https://issues.apache.org/jira/browse/SPARK-1755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Wendell updated SPARK-1755:
-----------------------------------

    Assignee: Andrew Or

> Spark-submit --name does not resolve to application name on YARN
> ----------------------------------------------------------------
>
>                 Key: SPARK-1755
>                 URL: https://issues.apache.org/jira/browse/SPARK-1755
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 0.9.1
>            Reporter: Andrew Or
>            Assignee: Andrew Or
>            Priority: Blocker
>             Fix For: 1.0.0
>
>
> In YARN client mode, --name is ignored because the deploy mode is client, and the name
is for some reason a [cluster config|https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L170)].
> In YARN cluster mode, --name is passed to the org.apache.spark.deploy.yarn.Client as
a command line argument. The Client class, however, uses this name only as the [app name for
the RM|https://github.com/apache/spark/blob/master/yarn/stable/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L80],
but not for Spark. In other words, when SparkConf attempts to load default configs, application
name is not set.
> In both cases, passing --name to SparkSubmit does not actually cause Spark to adopt it
as its application name, despite what the usage promises.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message