Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 4 Jan 2017 22:42:58 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13031721.1483467810000.644100.1483569778575@Atlassian.JIRA>
In-Reply-To: <JIRA.13031721.1483467810000@Atlassian.JIRA>
References: <JIRA.13031721.1483467810000@Atlassian.JIRA> <JIRA.13031721.1483467810882@arcas>
Subject: [jira] [Commented] (YARN-6050) AMs can't be scheduled on racks or
 nodes
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 04 Jan 2017 22:43:00 -0000


    [ https://issues.apache.org/jira/browse/YARN-6050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15799582#comment-15799582 ] 

Wangda Tan commented on YARN-6050:
----------------------------------

[~templedf], it is not only for AM, all existing ResourceRequest treats ResourceRequest with resourceName=* separately. Please file a ticket to track documentation / javadocs changes 

I think we need some additional checks / logics
- First, for AM request list, we need to make sure it has a valid offswitch request
- Second, as mentioned by Daniel, it's better to check asks resources and node-label-expression, if non-offswitch request has different resourceName/node-label-expression, we should set it to same as offswitch request
- Third, I think we still need to force priority and #asks of AM request, there could be some existing logics assume AM request has a special priority.

It's better to add a test to make sure check/validation/normalization of RMAppManager works as expected.

Thoughts? 

> AMs can't be scheduled on racks or nodes
> ----------------------------------------
>
>                 Key: YARN-6050
>                 URL: https://issues.apache.org/jira/browse/YARN-6050
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.9.0, 3.0.0-alpha2
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>         Attachments: YARN-6050.001.patch, YARN-6050.002.patch
>
>
> Yarn itself supports rack/node aware scheduling for AMs; however, there currently are two problems:
> # To specify hard or soft rack/node requests, you have to specify more than one {{ResourceRequest}}.  For example, if you want to schedule an AM only on "rackA", you have to create two {{ResourceRequest}}, like this:
> {code}
> ResourceRequest.newInstance(PRIORITY, ANY, CAPABILITY, NUM_CONTAINERS, false);
> ResourceRequest.newInstance(PRIORITY, "rackA", CAPABILITY, NUM_CONTAINERS, true);
> {code}
> The problem is that the Yarn API doesn't actually allow you to specify more than one {{ResourceRequest}} in the {{ApplicationSubmissionContext}}.  The current behavior is to either build one from {{getResource}} or directly from {{getAMContainerResourceRequest}}, depending on if {{getAMContainerResourceRequest}} is null or not.  We'll need to add a third method, say {{getAMContainerResourceRequests}}, which takes a list of {{ResourceRequest}} so that clients can specify the multiple resource requests.
> # There are some places where things are hardcoded to overwrite what the client specifies.  These are pretty straightforward to fix.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org