hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <ka...@cloudera.com>
Subject Re: Distributed shell with host affinity and relaxLocality
Date Mon, 16 Mar 2015 20:28:46 GMT
Hey Chris

What scheduler/version is this?

On Mon, Mar 16, 2015 at 12:01 PM, Chris Riccomini <
criccomini@linkedin.com.invalid> wrote:

> Hey all,
> We have been testing YARN with host-specific ContainerRequests. For our
> tests, we've been using the DistributedShell example. We've applied
> YARN-1974, which allows us to specify node lists, relax locality, etc.
> Everything seems to work as expected when we have relaxLocality set to
> false, and we request a specific host.
> When we set relaxLocality to true, things get weird. We run three nodes:
> node1, node2, and node3. When we start DistributedShell with, we configure
> it (via CLI params) to use two containers, and have a host-level request
> for node3. What we observe is that the AM and one container both end up on
> node2, and a third container ends up on node3. There are enough resources
> for node3 to handle both containers, but the second one doesn't end up
> there. We also notice that the DistributedShell app wedges because the
> container on node3 never completes.
> What is the expected behavior here? This seems to be broken.
> Cheers,
> Chris

Karthik Kambatla
Software Engineer, Cloudera Inc.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message