hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: Issues with ResourceManager scheduling functions
Date Mon, 09 Dec 2013 07:53:29 GMT
Are you requesting the different container types at different priorities?

-Sandy


On Sun, Dec 8, 2013 at 8:52 PM, Bill Q <bill.q.hdp@gmail.com> wrote:

> Hi,
> I recently implemented an application using YARN. Everything works fine,
> except for the illogical scheduling behavior of the ResourceManger, which
> might be caused by the heterogeneity of the cluster.
>
> I have two types of computing node in the cluster. 10 nodes with 128G
> memory and 10 nodes with 64G memory. In the application, I create two types
> of container. Type A container would ask for 50G of memory and Type B
> container would ask for 100G of memory.
>
> The problem is that if I created 50 Type B containers (100G), and 50 Type
> A containers (50G), Only Type B container (100G) are running. Instead of
> running the Type A container (50G), all the nodes with 64G memory are
> idling. And only the nodes with 128G memory are running Type B container
> (100G). This doesn't make too much sense to me.
>
> If I only created Type A containers (50G), all the nodes would be able run
> the containers. The 128G node would run two containers simultaneously. So,
> the problem seems to be connected to the different types of containers.
>
> Does anybody have any suggestions to fix this issue?
>
> Many thanks.
>
>
> Bill
>

Mime
View raw message