hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ravi Prakash <ravihad...@gmail.com>
Subject Re: Capacity scheduler for yarn oin 2.7.3 - problem with job scheduling to created queue.
Date Tue, 08 Nov 2016 19:51:06 GMT
Hi Rafal!

Have you been able to launch the job successfully first without configuring
node-labels? Do you really need node-labels? How much total memory do you
have on the cluster? Node labels are usually for specifying special
capabilities of the nodes (e.g. some nodes could have GPUs and your
application could request to be run on only the nodes which have GPUs)

HTH
Ravi

On Tue, Nov 8, 2016 at 6:19 AM, RafaƂ Radecki <radecki.rafal@gmail.com>
wrote:

> Hi All.
>
> I configured yarn to use capacity scheduler, I have for physical nodes. On
> first of them I run resourcemanager and nodemanager on all of them.
>
> My capacity-scheduler.xml and yarn-site.yml are attached.
> When I submit a job to the "long" queue I get in resourcemanager's logfile
> the content of attached rm.log and in RM qui the job is in state
> "ACCEPTED: waiting for AM container to be allocated, launched and
> register with RM"
> and has finalstatus
> "UNDEFINED"
>
> At the same time I see that in RM gui in scheduler section I have four
> partitions (node1-4d) in which in every one there are two queues "long" and
> "short" available. To summarize when I run a task, samza task in my case, I
> only specify (https://samza.apache.org/learn/documentation/0.10/jobs/
> yarn-jobs.html)
> yarn.queue=long
> or
> yarn.queue=short
>
> Have I missed something?
>
> BR,
> Rafal.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>

Mime
View raw message