spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sushrut Ikhar <sushrutikha...@gmail.com>
Subject Running Spark in Yarn-client mode
Date Thu, 08 Oct 2015 05:23:10 GMT
Hi,
I am new to Spark and I have been trying to run Spark in yarn-client mode.

I get this error in yarn logs :
Error: Could not find or load main class
org.apache.spark.executor.CoarseGrainedExecutorBackend

Also, I keep getting these warnings:

WARN YarnScheduler: Initial job has not accepted any resources; check your
cluster UI to ensure that workers are registered and have sufficient
resources

WARN YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster has
disassociated

WARN ReliableDeliverySupervisor: Association with remote system
[akka.tcp://sparkYarnAM@<hostname:port>] has failed, address is now gated
for [5000] ms. Reason is: [Disassociated].

I believe that executors are starting but are unable to connect back to the
driver.
How do I resolve this?
Also, I need help in locating the driver and executor node logs.

Thanks.

Regards,

Sushrut Ikhar
[image: https://]about.me/sushrutikhar
<https://about.me/sushrutikhar?promo=email_sig>

Mime
View raw message