hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit.vijayar...@gmail.com>
Subject Re: Uneven distribution of container allocations
Date Fri, 14 Dec 2012 18:22:49 GMT
I should have mentioned. I was trying this on FairScheduler.
I can try CapacityScheduler and see if that works.
But this is pretty easy to reproduce as I described in the problem.

2012/12/14 Harsh J <harsh@cloudera.com>

> Perhaps you can try YARN-80's
> yarn.scheduler.capacity.node-locality-delay set to 1 for your
> CapacityScheduler (are you using this or the Fifo)?
>
> On Fri, Dec 14, 2012 at 1:37 AM, lohit <lohit.vijayarenu@gmail.com> wrote:
> > I am running hadoop-2.0.3 alpha branch.
> > Right now I am trying TestDFSIO, where I would like to have even
> > distribution of tasks.
> > As soon as I submit job RM tends to allocate as many containers as
> possible
> > from subset of nodes.
> >
> > 2012/12/13 Thomas Graves <tgraves@yahoo-inc.com>
> >
> >> What kind of job are you running?  What version of YARN?
> >>
> >> Tom
> >>
> >> On 12/13/12 12:24 AM, "lohit" <lohit.vijayarenu@gmail.com> wrote:
> >>
> >> >Hi Devs,
> >> >
> >> >I am running few experiments on YARN. On a 50 node cluster I run only
> one
> >> >job.
> >> >If I kick off job with 150 tasks, I would expect on an average 3 tasks
> in
> >> >containers per node.
> >> >But I see many nodes end up having 29-30 containers spawned and many
> >> >without any tasks at all.
> >> >Do I have to use some kind of parameter (delayed scheduling) to achieve
> >> >even distribution?
> >> >
> >> >--
> >> >Have a Nice Day!
> >> >Lohit
> >>
> >>
> >
> >
> > --
> > Have a Nice Day!
> > Lohit
>
>
>
> --
> Harsh J
>



-- 
Have a Nice Day!
Lohit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message