spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcelo Vanzin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-13955) Spark in yarn mode fails
Date Thu, 17 Mar 2016 16:59:33 GMT

    [ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15199927#comment-15199927
] 

Marcelo Vanzin commented on SPARK-13955:
----------------------------------------

How did you build Spark? Did you do "mvn package" or "sbt assembly"? I see the code is trying
to uploade "file:/Users/jzhang/github/spark/lib/", do you have a file called RELEASE on your
repo's root? (That triggers a different code path, since that file is only expected to exist
in a Spark distribution, not in a Spark source repo.)

> Spark in yarn mode fails
> ------------------------
>
>                 Key: SPARK-13955
>                 URL: https://issues.apache.org/jira/browse/SPARK-13955
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Jeff Zhang
>
> I ran spark-shell in yarn client, but from the logs seems the spark assembly jar is not
uploaded to HDFS. This may be known issue in the process of SPARK-11157, create this ticket
to track this issue. [~vanzin]
> {noformat}
> 16/03/17 17:57:48 INFO Client: Will allocate AM container, with 896 MB memory including
384 MB overhead
> 16/03/17 17:57:48 INFO Client: Setting up container launch context for our AM
> 16/03/17 17:57:48 INFO Client: Setting up the launch environment for our AM container
> 16/03/17 17:57:48 INFO Client: Preparing resources for our AM container
> 16/03/17 17:57:48 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set,
falling back to uploading libraries under SPARK_HOME.
> 16/03/17 17:57:48 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.10.jar
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/apache-rat-0.10.jar
> 16/03/17 17:57:49 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.11.jar
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/apache-rat-0.11.jar
> 16/03/17 17:57:49 INFO Client: Uploading resource file:/private/var/folders/dp/hmchg5dd3vbcvds26q91spdw0000gp/T/spark-abed04bf-6ac2-448b-91a9-dcc1c401a18f/__spark_conf__4163776487351314654.zip
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/__spark_conf__4163776487351314654.zip
> 16/03/17 17:57:49 INFO SecurityManager: Changing view acls to: jzhang
> 16/03/17 17:57:49 INFO SecurityManager: Changing modify acls to: jzhang
> 16/03/17 17:57:49 INFO SecurityManager: SecurityManager: authentication disabled; ui
acls disabled; users with view permissions: Set(jzhang); users with modify permissions: Set(jzhang)
> 16/03/17 17:57:49 INFO Client: Submitting application 6 to ResourceManager
> {noformat}
> message in AM container
> {noformat}
> Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message