hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: Uneven distribution of container allocations
Date Fri, 14 Dec 2012 18:03:17 GMT
Perhaps you can try YARN-80's
yarn.scheduler.capacity.node-locality-delay set to 1 for your
CapacityScheduler (are you using this or the Fifo)?

On Fri, Dec 14, 2012 at 1:37 AM, lohit <lohit.vijayarenu@gmail.com> wrote:
> I am running hadoop-2.0.3 alpha branch.
> Right now I am trying TestDFSIO, where I would like to have even
> distribution of tasks.
> As soon as I submit job RM tends to allocate as many containers as possible
> from subset of nodes.
>
> 2012/12/13 Thomas Graves <tgraves@yahoo-inc.com>
>
>> What kind of job are you running?  What version of YARN?
>>
>> Tom
>>
>> On 12/13/12 12:24 AM, "lohit" <lohit.vijayarenu@gmail.com> wrote:
>>
>> >Hi Devs,
>> >
>> >I am running few experiments on YARN. On a 50 node cluster I run only one
>> >job.
>> >If I kick off job with 150 tasks, I would expect on an average 3 tasks in
>> >containers per node.
>> >But I see many nodes end up having 29-30 containers spawned and many
>> >without any tasks at all.
>> >Do I have to use some kind of parameter (delayed scheduling) to achieve
>> >even distribution?
>> >
>> >--
>> >Have a Nice Day!
>> >Lohit
>>
>>
>
>
> --
> Have a Nice Day!
> Lohit



-- 
Harsh J

Mime
View raw message