hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4819) SLS requested containers are never allocated for NODE_LOCAL and RACK_LOCAL type
Date Tue, 15 Mar 2016 08:09:33 GMT

    [ https://issues.apache.org/jira/browse/YARN-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15194891#comment-15194891
] 

Rohith Sharma K S commented on YARN-4819:
-----------------------------------------

In the method {{SLSUtils#getRackHostName}}, rack names are returned without slash. But this
causes ResourceRequest for resources rack has sent without slash.
{code}
@Private
@Unstable
public class SLSUtils {

  // hostname includes the network path and the host name. for example
  // "/default-rack/hostFoo" or "/coreSwitchA/TORSwitchB/hostBar".
  // the function returns two Strings, the first element is the network
  // location without "/", the second element is the host name. for example,
  // {"default-rack", "hostFoo"} or "coreSwitchA/TORSwitchB", "hostBar"
  public static String[] getRackHostName(String hostname) {
    NodeBase node = new NodeBase(hostname);
    return new String[] {node.getNetworkLocation().substring(1),
        node.getName()};
  }
{code}

> SLS requested containers are never allocated for NODE_LOCAL and RACK_LOCAL type
> -------------------------------------------------------------------------------
>
>                 Key: YARN-4819
>                 URL: https://issues.apache.org/jira/browse/YARN-4819
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: scheduler-load-simulator
>            Reporter: Rohith Sharma K S
>
> SLS requested containers are always assigned in off-switch mode. And scheduler will never
allocate any containers for Node-Local and Rack-local mode. 
> The reason for the above behavior is parsing of jobinput file for container host and
rack. The input given in job file for rack and node is like below.
> {code}"container.host" : "/default-rack/node1",{code}. But after parsing , the resource
request sent for rack is "default-rack" which misses slash('/'). While container allocation
during node heartbeat , node rack  name is resolved for rack which is */default-rack* with
which look up for resource request for this rack returns null. So, All the requests are assigned
to off-switch.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message