hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Q <bill.q....@gmail.com>
Subject Re: Issues with ResourceManager scheduling functions
Date Mon, 09 Dec 2013 08:23:53 GMT
No, all default 0. Should I use different priorities?

Many thanks.


On Mon, Dec 9, 2013 at 2:53 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:

> Are you requesting the different container types at different priorities?
> -Sandy
> On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <bill.q.hdp@gmail.com> wrote:
>> Hi,
>> I recently implemented an application using YARN. Everything works fine,
>> except for the illogical scheduling behavior of the ResourceManger, which
>> might be caused by the heterogeneity of the cluster.
>> I have two types of computing node in the cluster. 10 nodes with 128G
>> memory and 10 nodes with 64G memory. In the application, I create two types
>> of container. Type A container would ask for 50G of memory and Type B
>> container would ask for 100G of memory.
>> The problem is that if I created 50 Type B containers (100G), and 50 Type
>> A containers (50G), Only Type B container (100G) are running. Instead of
>> running the Type A container (50G), all the nodes with 64G memory are
>> idling. And only the nodes with 128G memory are running Type B container
>> (100G). This doesn't make too much sense to me.
>> If I only created Type A containers (50G), all the nodes would be able
>> run the containers. The 128G node would run two containers simultaneously.
>> So, the problem seems to be connected to the different types of containers.
>> Does anybody have any suggestions to fix this issue?
>> Many thanks.
>> Bill

View raw message