hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com.INVALID>
Subject Re: Distributed shell with host affinity and relaxLocality
Date Mon, 16 Mar 2015 20:31:06 GMT
+ Navina

Hey Karthik,

YARN 2.6.0 FairShare.


On 3/16/15 1:28 PM, "Karthik Kambatla" <kasha@cloudera.com> wrote:

>Hey Chris
>What scheduler/version is this?
>On Mon, Mar 16, 2015 at 12:01 PM, Chris Riccomini <
>criccomini@linkedin.com.invalid> wrote:
>> Hey all,
>> We have been testing YARN with host-specific ContainerRequests. For our
>> tests, we've been using the DistributedShell example. We've applied
>> YARN-1974, which allows us to specify node lists, relax locality, etc.
>> Everything seems to work as expected when we have relaxLocality set to
>> false, and we request a specific host.
>> When we set relaxLocality to true, things get weird. We run three nodes:
>> node1, node2, and node3. When we start DistributedShell with, we
>> it (via CLI params) to use two containers, and have a host-level request
>> for node3. What we observe is that the AM and one container both end up
>> node2, and a third container ends up on node3. There are enough
>> for node3 to handle both containers, but the second one doesn't end up
>> there. We also notice that the DistributedShell app wedges because the
>> container on node3 never completes.
>> What is the expected behavior here? This seems to be broken.
>> Cheers,
>> Chris
>Karthik Kambatla
>Software Engineer, Cloudera Inc.

View raw message