hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Graves <tgra...@yahoo-inc.com>
Subject Re: Uneven distribution of container allocations
Date Fri, 14 Dec 2012 15:26:15 GMT
We've been using branch-0.23 and haven't seen those issues. I'll give it a
try on branch-2.

Tom

On 12/13/12 2:07 PM, "lohit" <lohit.vijayarenu@gmail.com> wrote:

>I am running hadoop-2.0.3 alpha branch.
>Right now I am trying TestDFSIO, where I would like to have even
>distribution of tasks.
>As soon as I submit job RM tends to allocate as many containers as
>possible
>from subset of nodes.
>
>2012/12/13 Thomas Graves <tgraves@yahoo-inc.com>
>
>> What kind of job are you running?  What version of YARN?
>>
>> Tom
>>
>> On 12/13/12 12:24 AM, "lohit" <lohit.vijayarenu@gmail.com> wrote:
>>
>> >Hi Devs,
>> >
>> >I am running few experiments on YARN. On a 50 node cluster I run only
>>one
>> >job.
>> >If I kick off job with 150 tasks, I would expect on an average 3 tasks
>>in
>> >containers per node.
>> >But I see many nodes end up having 29-30 containers spawned and many
>> >without any tasks at all.
>> >Do I have to use some kind of parameter (delayed scheduling) to achieve
>> >even distribution?
>> >
>> >--
>> >Have a Nice Day!
>> >Lohit
>>
>>
>
>
>-- 
>Have a Nice Day!
>Lohit


Mime
View raw message