Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 27 Nov 2017 05:31:00 +0000 (UTC)
From: "wuchang (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13120833.1511759955000.321669.1511760660083@Atlassian.JIRA>
In-Reply-To: <JIRA.13120833.1511759955000@Atlassian.JIRA>
References: <JIRA.13120833.1511759955000@Atlassian.JIRA> <JIRA.13120833.1511759955187@jira-lw-us.apache.org>
Subject: [jira] [Updated] (YARN-7561) Why hasContainerForNode return false
 directly when there is no request of ANY locality?
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 27 Nov 2017 05:31:05 -0000


     [ https://issues.apache.org/jira/browse/YARN-7561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

wuchang updated YARN-7561:
--------------------------
    Description: 
I am studying the FairScheduler source cod of yarn 2.7.3.
By the code of class FSAppAttempt:

{code}
  public boolean hasContainerForNode(Priority prio, FSSchedulerNode node) {

    ResourceRequest anyRequest = getResourceRequest(prio, ResourceRequest.ANY);  
    ResourceRequest rackRequest = getResourceRequest(prio, node.getRackName()); 
    ResourceRequest nodeRequest = getResourceRequest(prio, node.getNodeName()); 
    
    return
        // There must be outstanding requests at the given priority:
        anyRequest != null && anyRequest.getNumContainers() > 0 &&
            // If locality relaxation is turned off at *-level, there must be a
            // non-zero request for the node's rack:
            (anyRequest.getRelaxLocality() ||
                (rackRequest != null && rackRequest.getNumContainers() > 0)) &&
            // If locality relaxation is turned off at rack-level, there must be a
            // non-zero request at the node:
            (rackRequest == null || rackRequest.getRelaxLocality() ||
                (nodeRequest != null && nodeRequest.getNumContainers() > 0)) &&
            // The requested container must be able to fit on the node:
            Resources.lessThanOrEqual(RESOURCE_CALCULATOR, null,
                anyRequest.getCapability(), node.getRMNode().getTotalCapability());
}
{code}


I really cannot understand why when there is no anyRequest , *hasContainerForNode()* return false directly without considering whether there is NODE_LOCAL  or  RACK_LOCAL requests.
And ,  *AppSchedulingInfo.allocateNodeLocal()* and *AppSchedulingInfo.allocateRackLocal()* will also decrease the number of containers for *ResourceRequest.ANY*, this is another place where I feel confused.

Really thanks for some prompt.

  was:
I am studying the FairScheduler source cod of yarn 2.7.3.
By the code of class FSAppAttempt:

{quote}
  public boolean hasContainerForNode(Priority prio, FSSchedulerNode node) {

    ResourceRequest anyRequest = getResourceRequest(prio, ResourceRequest.ANY);  
    ResourceRequest rackRequest = getResourceRequest(prio, node.getRackName()); 
    ResourceRequest nodeRequest = getResourceRequest(prio, node.getNodeName()); 
    
    return
        // There must be outstanding requests at the given priority:
        anyRequest != null && anyRequest.getNumContainers() > 0 &&
            // If locality relaxation is turned off at *-level, there must be a
            // non-zero request for the node's rack:
            (anyRequest.getRelaxLocality() ||
                (rackRequest != null && rackRequest.getNumContainers() > 0)) &&
            // If locality relaxation is turned off at rack-level, there must be a
            // non-zero request at the node:
            (rackRequest == null || rackRequest.getRelaxLocality() ||
                (nodeRequest != null && nodeRequest.getNumContainers() > 0)) &&
            // The requested container must be able to fit on the node:
            Resources.lessThanOrEqual(RESOURCE_CALCULATOR, null,
                anyRequest.getCapability(), node.getRMNode().getTotalCapability());
}
{quote}


I really cannot understand why when there is no anyRequest , *hasContainerForNode()* return false directly without considering whether there is NODE_LOCAL  or  RACK_LOCAL requests.
And ,  *AppSchedulingInfo.allocateNodeLocal()* and *AppSchedulingInfo.allocateRackLocal()* will also decrease the number of containers for *ResourceRequest.ANY*, this is another place where I feel confused.

Really thanks for some prompt.


> Why hasContainerForNode return false directly when there is no request of ANY locality?
> ---------------------------------------------------------------------------------------
>
>                 Key: YARN-7561
>                 URL: https://issues.apache.org/jira/browse/YARN-7561
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: fairscheduler
>    Affects Versions: 2.7.3
>            Reporter: wuchang
>
> I am studying the FairScheduler source cod of yarn 2.7.3.
> By the code of class FSAppAttempt:
> {code}
>   public boolean hasContainerForNode(Priority prio, FSSchedulerNode node) {
>     ResourceRequest anyRequest = getResourceRequest(prio, ResourceRequest.ANY);  
>     ResourceRequest rackRequest = getResourceRequest(prio, node.getRackName()); 
>     ResourceRequest nodeRequest = getResourceRequest(prio, node.getNodeName()); 
>     
>     return
>         // There must be outstanding requests at the given priority:
>         anyRequest != null && anyRequest.getNumContainers() > 0 &&
>             // If locality relaxation is turned off at *-level, there must be a
>             // non-zero request for the node's rack:
>             (anyRequest.getRelaxLocality() ||
>                 (rackRequest != null && rackRequest.getNumContainers() > 0)) &&
>             // If locality relaxation is turned off at rack-level, there must be a
>             // non-zero request at the node:
>             (rackRequest == null || rackRequest.getRelaxLocality() ||
>                 (nodeRequest != null && nodeRequest.getNumContainers() > 0)) &&
>             // The requested container must be able to fit on the node:
>             Resources.lessThanOrEqual(RESOURCE_CALCULATOR, null,
>                 anyRequest.getCapability(), node.getRMNode().getTotalCapability());
> }
> {code}
> I really cannot understand why when there is no anyRequest , *hasContainerForNode()* return false directly without considering whether there is NODE_LOCAL  or  RACK_LOCAL requests.
> And ,  *AppSchedulingInfo.allocateNodeLocal()* and *AppSchedulingInfo.allocateRackLocal()* will also decrease the number of containers for *ResourceRequest.ANY*, this is another place where I feel confused.
> Really thanks for some prompt.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org