Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of lohit.vijayarenu@gmail.com
 designates 209.85.214.176 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CACBYxKLoG-uh-2B0ks5nFqeq5SCQ6+nCU7TuLnbjbUF52YqRKA@mail.gmail.com>
References: 
 <CA+01ahh02RfByEEb7wRE7SkC8s01TEGuUKVYsgySmELpy0AkSQ@mail.gmail.com>
	<CCEF8FF5.2B876%tgraves@yahoo-inc.com>
	<CA+01ahgsKemxWBOrj5uojWQTbS5_f9rxUx-HqeiY7DiaUTt+NQ@mail.gmail.com>
	<CAOcnVr2vtiJztitBHnQU3khPGjTUNrqmuzyBF6qKb-UL3T9Nfg@mail.gmail.com>
	<CA+01ahhPgsE6=FgHg2Lcsc6ibHUhaSHvn_qx8mhppGf5mPHtBA@mail.gmail.com>
	<CACBYxKLoG-uh-2B0ks5nFqeq5SCQ6+nCU7TuLnbjbUF52YqRKA@mail.gmail.com>
Date: Fri, 14 Dec 2012 15:26:48 -0800
Message-ID: 
 <CA+01ahiS_hNgibYHhD8pUngpx0zkT0A2R0c_LthJ1Z5j63ujGw@mail.gmail.com>
Subject: Re: Uneven distribution of container allocations
From: lohit <lohit.vijayarenu@gmail.com>
To: yarn-dev@hadoop.apache.org
Content-Type: multipart/alternative; boundary=e89a8f6438bc9626b304d0d85df1

--e89a8f6438bc9626b304d0d85df1
Content-Type: text/plain; charset=UTF-8

Thanks Sandy, this worked.

2012/12/14 Sandy Ryza <sandy.ryza@cloudera.com>

> Hi Lohit,
>
> Setting yarn.scheduler.fair.assignmultiple to false should give a more even
> distribution of tasks.
>
> -Sandy
>
> On Fri, Dec 14, 2012 at 10:22 AM, lohit <lohit.vijayarenu@gmail.com>
> wrote:
>
> > I should have mentioned. I was trying this on FairScheduler.
> > I can try CapacityScheduler and see if that works.
> > But this is pretty easy to reproduce as I described in the problem.
> >
> > 2012/12/14 Harsh J <harsh@cloudera.com>
> >
> > > Perhaps you can try YARN-80's
> > > yarn.scheduler.capacity.node-locality-delay set to 1 for your
> > > CapacityScheduler (are you using this or the Fifo)?
> > >
> > > On Fri, Dec 14, 2012 at 1:37 AM, lohit <lohit.vijayarenu@gmail.com>
> > wrote:
> > > > I am running hadoop-2.0.3 alpha branch.
> > > > Right now I am trying TestDFSIO, where I would like to have even
> > > > distribution of tasks.
> > > > As soon as I submit job RM tends to allocate as many containers as
> > > possible
> > > > from subset of nodes.
> > > >
> > > > 2012/12/13 Thomas Graves <tgraves@yahoo-inc.com>
> > > >
> > > >> What kind of job are you running?  What version of YARN?
> > > >>
> > > >> Tom
> > > >>
> > > >> On 12/13/12 12:24 AM, "lohit" <lohit.vijayarenu@gmail.com> wrote:
> > > >>
> > > >> >Hi Devs,
> > > >> >
> > > >> >I am running few experiments on YARN. On a 50 node cluster I run
> only
> > > one
> > > >> >job.
> > > >> >If I kick off job with 150 tasks, I would expect on an average 3
> > tasks
> > > in
> > > >> >containers per node.
> > > >> >But I see many nodes end up having 29-30 containers spawned and
> many
> > > >> >without any tasks at all.
> > > >> >Do I have to use some kind of parameter (delayed scheduling) to
> > achieve
> > > >> >even distribution?
> > > >> >
> > > >> >--
> > > >> >Have a Nice Day!
> > > >> >Lohit
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > Have a Nice Day!
> > > > Lohit
> > >
> > >
> > >
> > > --
> > > Harsh J
> > >
> >
> >
> >
> > --
> > Have a Nice Day!
> > Lohit
> >
>


-- 
Have a Nice Day!
Lohit

--e89a8f6438bc9626b304d0d85df1--