hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "gaurav gupta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1412) Allocating Containers on a particular Node in Yarn
Date Thu, 21 Nov 2013 01:23:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-1412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13828368#comment-13828368

gaurav gupta commented on YARN-1412:

 yarn.scheduler.capacity.node-locality-delay is set to 50 and the size of cluster is 36. 
Since this property affects the cluster, not sure if it is right thing to depend on this property
for container allocation

For the nature of our application we are requesting containers with incremental priorities
so we have 1 container per priority, so we can't request for multiple containers at pri 0
and there are some applications were total number of containers are less than the cluster

We want what you mentioned "Btw, what you are trying to do (node=specific, rack=null and relaxLocality=true)
is the default behavior of existing schedulers. They will always try to relax locality to
rack and then off-switch by default. So you dont need to explicitly code for it. "... But
since it is not working we are trying to code it.

> Allocating Containers on a particular Node in Yarn
> --------------------------------------------------
>                 Key: YARN-1412
>                 URL: https://issues.apache.org/jira/browse/YARN-1412
>             Project: Hadoop YARN
>          Issue Type: Bug
>         Environment: centos, Hadoop 2.2.0
>            Reporter: gaurav gupta
> Summary of the problem: 
>  If I pass the node on which I want container and set relax locality default which is
true, I don't get back the container on the node specified even if the resources are available
on the node. It doesn't matter if I set rack or not.
> Here is the snippet of the code that I am using
> AMRMClient<ContainerRequest> amRmClient =  AMRMClient.createAMRMClient();;
>     String host = "h1";
>     Resource capability = Records.newRecord(Resource.class);
>     capability.setMemory(memory);
>     nodes = new String[] {host};
>     // in order to request a host, we also have to request the rack
>     racks = new String[] {"/default-rack"};
>      List<ContainerRequest> containerRequests = new ArrayList<ContainerRequest>();
>     List<ContainerId> releasedContainers = new ArrayList<ContainerId>();
>     containerRequests.add(new ContainerRequest(capability, nodes, racks, Priority.newInstance(priority)));
>     if (containerRequests.size() > 0) {
>       LOG.info("Asking RM for containers: " + containerRequests);
>       for (ContainerRequest cr : containerRequests) {
>         LOG.info("Requested container: {}", cr.toString());
>         amRmClient.addContainerRequest(cr);
>       }
>     }
>     for (ContainerId containerId : releasedContainers) {
>       LOG.info("Released container, id={}", containerId.getId());
>       amRmClient.releaseAssignedContainer(containerId);
>     }
>     return amRmClient.allocate(0);

This message was sent by Atlassian JIRA

View raw message