hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Q <bill.q....@gmail.com>
Subject Issues with ResourceManager scheduling functions
Date Mon, 09 Dec 2013 04:52:13 GMT
I recently implemented an application using YARN. Everything works fine,
except for the illogical scheduling behavior of the ResourceManger, which
might be caused by the heterogeneity of the cluster.

I have two types of computing node in the cluster. 10 nodes with 128G
memory and 10 nodes with 64G memory. In the application, I create two types
of container. Type A container would ask for 50G of memory and Type B
container would ask for 100G of memory.

The problem is that if I created 50 Type B containers (100G), and 50 Type A
containers (50G), Only Type B container (100G) are running. Instead of
running the Type A container (50G), all the nodes with 64G memory are
idling. And only the nodes with 128G memory are running Type B container
(100G). This doesn't make too much sense to me.

If I only created Type A containers (50G), all the nodes would be able run
the containers. The 128G node would run two containers simultaneously. So,
the problem seems to be connected to the different types of containers.

Does anybody have any suggestions to fix this issue?

Many thanks.


View raw message