spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject Re: Spark on YARN
Date Tue, 18 Nov 2014 18:08:53 GMT
Can you check in your RM's web UI how much of each resource does Yarn
think you have available? You can also check that in the Yarn
configuration directly.

Perhaps it's not configured to use all of the available resources. (If
it was set up with Cloudera Manager, CM will reserve some room for
daemons that need to run on each machine, so it won't tell Yarn to
make all 32 cores / 64 GB available for applications.)

Also remember that Spark needs to start "num executors + 1" containers
when adding up all the needed resources. The extra container generally
requires less resources than the executors, but it still needs to
allocate resources from the RM.



On Tue, Nov 18, 2014 at 10:03 AM, Alan Prando <alan@scanboo.com.br> wrote:
> Hi Folks!
>
> I'm running Spark on YARN cluster installed with Cloudera Manager Express.
> The cluster has 1 master and 3 slaves, each machine with 32 cores and 64G
> RAM.
>
> My spark's job is working fine, however it seems that just 2 of 3 slaves are
> working (htop shows 2 slaves working 100% on 32 cores, and 1 slaves without
> any processing).
>
> I'm using this command:
> ./spark-submit --master yarn --num-executors 3 --executor-cores 32
> --executor-memory 32g feature_extractor.py -r 390
>
> Additionaly, spark's log testify communications with 2 slaves only:
> 14/11/18 17:19:38 INFO YarnClientSchedulerBackend: Registered executor:
> Actor[akka.tcp://sparkExecutor@ip-172-31-13-180.ec2.internal:33177/user/Executor#-113177469]
> with ID 1
> 14/11/18 17:19:38 INFO RackResolver: Resolved ip-172-31-13-180.ec2.internal
> to /default
> 14/11/18 17:19:38 INFO YarnClientSchedulerBackend: Registered executor:
> Actor[akka.tcp://sparkExecutor@ip-172-31-13-179.ec2.internal:51859/user/Executor#-323896724]
> with ID 2
> 14/11/18 17:19:38 INFO RackResolver: Resolved ip-172-31-13-179.ec2.internal
> to /default
> 14/11/18 17:19:38 INFO BlockManagerMasterActor: Registering block manager
> ip-172-31-13-180.ec2.internal:50959 with 16.6 GB RAM
> 14/11/18 17:19:39 INFO BlockManagerMasterActor: Registering block manager
> ip-172-31-13-179.ec2.internal:53557 with 16.6 GB RAM
> 14/11/18 17:19:51 INFO YarnClientSchedulerBackend: SchedulerBackend is ready
> for scheduling beginning after waiting maxRegisteredResourcesWaitingTime:
> 30000(ms)
>
> Is there a configuration to call spark's job on YARN cluster with all
> slaves?
>
> Thanks in advance! =]
>
> ---
> Regards
> Alan Vidotti Prando.
>
>



-- 
Marcelo

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message