hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com.INVALID>
Subject Distributed shell with host affinity and relaxLocality
Date Mon, 16 Mar 2015 19:01:33 GMT
Hey all,

We have been testing YARN with host-specific ContainerRequests. For our tests, we've been
using the DistributedShell example. We've applied YARN-1974, which allows us to specify node
lists, relax locality, etc. Everything seems to work as expected when we have relaxLocality
set to false, and we request a specific host.

When we set relaxLocality to true, things get weird. We run three nodes: node1, node2, and
node3. When we start DistributedShell with, we configure it (via CLI params) to use two containers,
and have a host-level request for node3. What we observe is that the AM and one container
both end up on node2, and a third container ends up on node3. There are enough resources for
node3 to handle both containers, but the second one doesn't end up there. We also notice that
the DistributedShell app wedges because the container on node3 never completes.

What is the expected behavior here? This seems to be broken.

Cheers,
Chris

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message