hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gaurav Gupta" <gau...@datatorrent.com>
Subject RE: Allocating Containers on a particular Node in Yarn
Date Fri, 15 Nov 2013 01:48:13 GMT
Hi Bikas,

With scheduler delay on and relax locality set to true (with and without
Requesting the rack), I don't get the containers on the required host. It
always assign to different host.
I am using default Capacity scheduler. Here is the snippet of the code

    AMRMClient<ContainerRequest> amRmClient =
AMRMClient.createAMRMClient();;
    String host = "h1";
    Resource capability = Records.newRecord(Resource.class);
    capability.setMemory(memory);
    nodes = new String[] {host};
    // in order to request a host, we also have to request the rack
    racks = new String[] {"/default-rack"};
     List<ContainerRequest> containerRequests = new
ArrayList<ContainerRequest>();
    List<ContainerId> releasedContainers = new ArrayList<ContainerId>();
    containerRequests.add(new ContainerRequest(capability, nodes, racks,
Priority.newInstance(priority),false));
    if (containerRequests.size() > 0) {
      LOG.info("Asking RM for containers: " + containerRequests);
      for (ContainerRequest cr : containerRequests) {
        LOG.info("Requested container: {}", cr.toString());
        amRmClient.addContainerRequest(cr);
      }
    }

    for (ContainerId containerId : releasedContainers) {
      LOG.info("Released container, id={}", containerId.getId());
      amRmClient.releaseAssignedContainer(containerId);
    }
    return amRmClient.allocate(0);



Thanks
Gaurav

-----Original Message-----
From: Bikas Saha [mailto:bikas@hortonworks.com] 
Sent: Wednesday, November 13, 2013 7:05 PM
To: yarn-dev@hadoop.apache.org
Subject: RE: Allocating Containers on a particular Node in Yarn

What you ask, try on request node and then fallback to others, is the
default behavior for current schedulers in yarn. Ie. relaxLocality is true
by default.

-----Original Message-----
From: Thomas Weise [mailto:thomas.weise@gmail.com]
Sent: Wednesday, November 13, 2013 3:55 PM
To: yarn-dev@hadoop.apache.org
Subject: Re: Allocating Containers on a particular Node in Yarn

Is it possible to specify a particular node and have RM fallback to an
different node only after making an attempt to allocate for the requested
node? In other words, is the combination of specific host name and
relaxLocality=TRUE meaningful at all?

Thanks.


On Wed, Nov 13, 2013 at 3:23 PM, Alejandro Abdelnur
<tucu@cloudera.com>wrote:

> Gaurav,
>
> Setting relaxLocality to FALSE should do it.
>
> thanks.
>
>
> On Wed, Nov 13, 2013 at 2:58 PM, gaurav <gaurav@datatorrent.com> wrote:
>
> >  Hi,
> >  I am trying to allocate containers on a particular node in Yarn but 
> > Yarn is returning me containers on different node although the 
> > requested node has resources available.
> >
> > I checked into the allocate(AllocateRequest request) function of 
> > ApplicationMasterService and my request is as follows
> >
> > *request: ask { priority { priority: 1 } resource_name: "h2"
> > capability {
> > memory: 1000 } num_containers: 2 } ask { priority { priority: 1 }
> > resource_name: "/default-rack" capability { memory: 1000 }
> num_containers:
> > 2 } ask { priority { priority: 1 } resource_name: "*" capability {
> memory:
> > 1000 } num_containers: 2 } response_id: 1 progress: 0.0*
> >
> > but the containers that I am getting back is as follows
> > [Container: [ContainerId: container_1384381084244_0001_01_000002,
NodeId:
> > h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
> > Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 
> > }, ],
> > Container: [ContainerId: container_1384381084244_0001_01_000003,
NodeId:
> > h1:1234, NodeHttpAddress: h1:2, Resource: <memory:1024, vCores:1>,
> > Priority: 1, Token: Token { kind: ContainerToken, service: h1:1234 
> > }, ]]
> >
> > I am attaching the test case that I have written along with the 
> > mail. It uses classes under
org.apache.hadoop.yarn.server.resourcemanager package.
> >
> > Any pointers would be of great help
> >
> > Thanks
> > Gaurav
> >
> >
> >
> >
> >
>
>
> --
> Alejandro
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.


Mime
View raw message