spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-13955) Spark in yarn mode fails
Date Sat, 19 Mar 2016 00:29:33 GMT

    [ https://issues.apache.org/jira/browse/SPARK-13955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15202411#comment-15202411
] 

Jeff Zhang edited comment on SPARK-13955 at 3/19/16 12:28 AM:
--------------------------------------------------------------

I can reproduce it with both sbt and mvn.  The scenario is same that spark assembly jar is
not uploaded to hdfs.  And there's no RELEASE file in my repo


was (Author: zjffdu):
I can reproduce it with both sbt and mvn.  The scenario is same that spark assembly jar is
not uploaded to hdfs. 

> Spark in yarn mode fails
> ------------------------
>
>                 Key: SPARK-13955
>                 URL: https://issues.apache.org/jira/browse/SPARK-13955
>             Project: Spark
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>            Reporter: Jeff Zhang
>
> I ran spark-shell in yarn client, but from the logs seems the spark assembly jar is not
uploaded to HDFS. This may be known issue in the process of SPARK-11157, create this ticket
to track this issue. [~vanzin]
> {noformat}
> 16/03/17 17:57:48 INFO Client: Will allocate AM container, with 896 MB memory including
384 MB overhead
> 16/03/17 17:57:48 INFO Client: Setting up container launch context for our AM
> 16/03/17 17:57:48 INFO Client: Setting up the launch environment for our AM container
> 16/03/17 17:57:48 INFO Client: Preparing resources for our AM container
> 16/03/17 17:57:48 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set,
falling back to uploading libraries under SPARK_HOME.
> 16/03/17 17:57:48 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.10.jar
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/apache-rat-0.10.jar
> 16/03/17 17:57:49 INFO Client: Uploading resource file:/Users/jzhang/github/spark/lib/apache-rat-0.11.jar
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/apache-rat-0.11.jar
> 16/03/17 17:57:49 INFO Client: Uploading resource file:/private/var/folders/dp/hmchg5dd3vbcvds26q91spdw0000gp/T/spark-abed04bf-6ac2-448b-91a9-dcc1c401a18f/__spark_conf__4163776487351314654.zip
-> hdfs://localhost:9000/user/jzhang/.sparkStaging/application_1458187008455_0006/__spark_conf__4163776487351314654.zip
> 16/03/17 17:57:49 INFO SecurityManager: Changing view acls to: jzhang
> 16/03/17 17:57:49 INFO SecurityManager: Changing modify acls to: jzhang
> 16/03/17 17:57:49 INFO SecurityManager: SecurityManager: authentication disabled; ui
acls disabled; users with view permissions: Set(jzhang); users with modify permissions: Set(jzhang)
> 16/03/17 17:57:49 INFO Client: Submitting application 6 to ResourceManager
> {noformat}
> message in AM container
> {noformat}
> Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message