spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sushrut Ikhar <sushrutikha...@gmail.com>
Subject Re: Running Spark in Yarn-client mode
Date Thu, 08 Oct 2015 06:12:40 GMT
Hey Jean,
Thanks for the quick response. I am using spark 1.4.1 pre-built with hadoop
2.6.
Yes the Yarn cluster has multiple running worker nodes.
It would a great help if you can tell how to look for the executors logs.

Regards,

Sushrut Ikhar
[image: https://]about.me/sushrutikhar
<https://about.me/sushrutikhar?promo=email_sig>


On Thu, Oct 8, 2015 at 11:22 AM, Jean-Baptiste Onofré <jb@nanthrax.net>
wrote:

> Hi Sushrut,
>
> which packaging of Spark do you use ?
> Do you have a working Yarn cluster (with at least one worker) ?
>
> spark-hadoop-x ?
>
> Regards
> JB
>
> On 10/08/2015 07:23 AM, Sushrut Ikhar wrote:
>
>> Hi,
>> I am new to Spark and I have been trying to run Spark in yarn-client mode.
>>
>> I get this error in yarn logs :
>> Error: Could not find or load main class
>> org.apache.spark.executor.CoarseGrainedExecutorBackend
>>
>> Also, I keep getting these warnings:
>>
>> WARN YarnScheduler: Initial job has not accepted any resources; check
>> your cluster UI to ensure that workers are registered and have
>> sufficient resources
>>
>> WARN YarnSchedulerBackend$YarnSchedulerEndpoint: ApplicationMaster has
>> disassociated
>>
>> WARN ReliableDeliverySupervisor: Association with remote system
>> [akka.tcp://sparkYarnAM@<hostname:port>] has failed, address is now
>> gated for [5000] ms. Reason is: [Disassociated].
>>
>> I believe that executors are starting but are unable to connect back to
>> the driver.
>> How do I resolve this?
>> Also, I need help in locating the driver and executor node logs.
>>
>> Thanks.
>>
>> Regards,
>>
>> Sushrut Ikhar
>> https://about.me/sushrutikhar
>>
>> <https://about.me/sushrutikhar?promo=email_sig>
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message