hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Issues with ResourceManager scheduling functions
Date Mon, 09 Dec 2013 08:30:30 GMT
YARN currently is unable to handle requests with different resource
requirements at the same priority (YARN-314).  Using different priorities
would likely solve the issue.  I believe it would also be safest to avoid
using the priority 0 because this is used for the application master.

-Sandy


On Mon, Dec 9, 2013 at 12:23 AM, Bill Q <bill.q.hdp@gmail.com> wrote:

> No, all default 0. Should I use different priorities?
>
> Many thanks.
>
>
> Bill
>
>
> On Mon, Dec 9, 2013 at 2:53 AM, Sandy Ryza <sandy.ryza@cloudera.com>wrote:
>
>> Are you requesting the different container types at different priorities?
>>
>> -Sandy
>>
>>
>> On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <bill.q.hdp@gmail.com> wrote:
>>
>>> Hi,
>>> I recently implemented an application using YARN. Everything works fine,
>>> except for the illogical scheduling behavior of the ResourceManger, which
>>> might be caused by the heterogeneity of the cluster.
>>>
>>> I have two types of computing node in the cluster. 10 nodes with 128G
>>> memory and 10 nodes with 64G memory. In the application, I create two types
>>> of container. Type A container would ask for 50G of memory and Type B
>>> container would ask for 100G of memory.
>>>
>>> The problem is that if I created 50 Type B containers (100G), and 50
>>> Type A containers (50G), Only Type B container (100G) are running. Instead
>>> of running the Type A container (50G), all the nodes with 64G memory are
>>> idling. And only the nodes with 128G memory are running Type B container
>>> (100G). This doesn't make too much sense to me.
>>>
>>> If I only created Type A containers (50G), all the nodes would be able
>>> run the containers. The 128G node would run two containers simultaneously.
>>> So, the problem seems to be connected to the different types of containers.
>>>
>>> Does anybody have any suggestions to fix this issue?
>>>
>>> Many thanks.
>>>
>>>
>>> Bill
>>>
>>
>>
>

Mime
View raw message