hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From devdoer bird <devd...@gmail.com>
Subject Re: why FairScheduler prefer to schedule MR jobs into the same node?
Date Sat, 10 Aug 2013 06:39:53 GMT
Thanks. Let me try this configuration.  I use the default  setting. Is it
set to true in default settings?


2013/8/10 Karthik Kambatla <kasha@cloudera.com>

> It is possible that you have assignMultiple set to true in your fair
> scheduler configuration - that leads to assigning as many tasks on a single
> node heartbeat as the node can accommodate. Setting it to false, would
> assign a single task on each heartbeat and can help in spreading out the
> tasks.
>
>
> On Fri, Aug 9, 2013 at 4:44 AM, devdoer bird <devdoer@gmail.com> wrote:
>
>> The hadoop version is 1.0.3
>>
>>
>> 2013/8/9 Sandy Ryza <sandy.ryza@cloudera.com>
>>
>>> Hi devdoer,
>>>
>>> What version are you using?
>>>
>>> -Sandy
>>>
>>>
>>> On Thu, Aug 8, 2013 at 4:25 AM, devdoer bird <devdoer@gmail.com> wrote:
>>>
>>>> HI:
>>>>
>>>> I configure the FairScheduler with default settings and my job has 19
>>>>  reduce tasks. I found that all the reduce tasks are schedule to run in one
>>>> node.
>>>>
>>>> While with default FIFO schedule, the 19 reduce tasks are scheduled
>>>> into diffrent nodes.
>>>>
>>>> How can I configure FairSchedule to load more balance?
>>>>
>>>>
>>>> --
>>>> devdoer
>>>> devdoer@gmail.com
>>>>
>>>
>>>
>>
>>
>> --
>> devdoer
>> devdoer@gmail.com
>>
>
>


-- 
devdoer
devdoer@gmail.com

Mime
View raw message