hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhijie Shen <zs...@hortonworks.com>
Subject Re: Getting Updated Node Report from AllocateResponse
Date Wed, 19 Feb 2014 19:29:42 GMT
Hi Gaurav,

Unfortunately, there's no async implementation that can push you the node
report when it is updated. What you can do now is to periodically invoke
the getNodeReports method in your code.

- Zhijie


On Wed, Feb 19, 2014 at 11:06 AM, Gaurav Gupta <gaurav@datatorrent.com>wrote:

> Hi Sandy,
>
> I need to get these updates periodically and not sure if current
> implementation (YarnClientImpl) of YarnClient is good for periodic updates.
> Are there any other async/light weight APIs?
>
> Gaurav
>
> -----Original Message-----
> From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
> Sent: Wednesday, February 12, 2014 3:20 PM
> To: yarn-dev@hadoop.apache.org
> Subject: Re: Getting Updated Node Report from AllocateResponse
>
> You'll need to use YarnClient to request this info directly from the
> ResourceManager.
>
>
> On Wed, Feb 12, 2014 at 3:08 PM, Gaurav Gupta <gaurav@datatorrent.com
> >wrote:
>
> > I am looking for the updated resources information like memory left,
> > Virtual Cores etc..
> >
> > -----Original Message-----
> > From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
> > Sent: Wednesday, February 12, 2014 1:19 PM
> > To: yarn-dev@hadoop.apache.org
> > Subject: Re: Getting Updated Node Report from AllocateResponse
> >
> > What specific information about those nodes are you looking for?
> >
> > -Sandy
> >
> >
> > On Wed, Feb 12, 2014 at 12:21 PM, Gaurav Gupta
> > <gaurav@datatorrent.com>wrote:
> >
> > > Thanks Sandy.. It helps..
> > > How do I get the updated node report for the nodes that have had
> > > containers allocated on them?
> > >
> > > Thanks
> > > Gaurav
> > >
> > > -----Original Message-----
> > > From: Sandy Ryza [mailto:sandy.ryza@cloudera.com]
> > > Sent: Wednesday, February 12, 2014 10:35 AM
> > > To: yarn-dev@hadoop.apache.org
> > > Subject: Re: Getting Updated Node Report from AllocateResponse
> > >
> > > Hi Guarav,
> > >
> > > The updated nodes reported in the AllocateResponse are not those
> > > that have had containers allocated on them recently for the
> application.
> > > Rather, they're the ones whose status in the cluster has changed, e.g.
> > > those that have recently connected, become unhealthy, or disconnected.
> > >
> > > hope that helps,
> > > Sandy
> > >
> > >
> > > On Tue, Feb 11, 2014 at 11:29 AM, Gaurav Gupta
> > > <gaurav@datatorrent.com>wrote:
> > >
> > > > I am using Hadoop 2.2.0 Version
> > > >
> > > > hadoop@node14:~$ hadoop version
> > > > Hadoop 2.2.0
> > > > Subversion https://svn.apache.org/repos/asf/hadoop/common -r
> > > > 1529768 Compiled by hortonmu on 2013-10-07T06:28Z Compiled with
> > > > protoc 2.5.0
> > > >
> > > > -----Original Message-----
> > > > From: Gaurav Gupta [mailto:gaurav@datatorrent.com]
> > > > Sent: Tuesday, February 11, 2014 11:16 AM
> > > > To: yarn-dev@hadoop.apache.org
> > > > Subject: Getting Updated Node Report from AllocateResponse
> > > >
> > > > Hi,
> > > >
> > > >
> > > >
> > > > I am trying to get the list of nodes whose status has been updated
> > > > from the AllocateResponse Obj, but I get back empty list although
> > > > containers are allocated to my App Master. Here is the code
> > > > snippet for the same
> > > >
> > > >
> > > >
> > > > for (ContainerRequest cr : containerRequests) {
> > > >
> > > >         LOG.info("Requested container: {}", cr.toString());
> > > >
> > > >         amRmClient.addContainerRequest(cr);
> > > >
> > > >       }
> > > >
> > > > for (ContainerId containerId : releasedContainers) {
> > > >
> > > >       LOG.info("Released container, id={}", containerId.getId());
> > > >
> > > >       amRmClient.releaseAssignedContainer(containerId);
> > > >
> > > >     }
> > > >
> > > > AllocateResponse amResp  = amRmClient.allocate(0);
> > > >
> > > > List<Container> newAllocatedContainers =
> > > > amResp.getAllocatedContainers();
> > > >
> > > > List<NodeReport> updatedNodes = amResp.getUpdatedNodes();
> > > >
> > > >
> > > >
> > > > I always empty list in the last call. Any pointers?
> > > >
> > > >
> > > >
> > > > Thanks
> > > >
> > > > -Gaurav
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> >
>
>


-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message