hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: When does the RM ever send out node map updates?
Date Mon, 19 Oct 2015 09:31:13 GMT

Thanks for that snippet.

Which brings me back to a key question: how does the AM determine the map of nodes.

And before someone says "the yarn client lets you do that", bear in mind this is for a long-lived
restartable AM; what may have been valid when the client launched it 3 weeks earlier may be
invalid on restart. 

If it were in the RegisterApplicationMasterResponse I'd pick it up there & wait for changes
—but you don't get a nodemap then

Any suggestions?

> On 19 Oct 2015, at 06:25, Rohith Sharma K S <rohithsharmaks@huawei.com> wrote:
> 
> NM node updates are sent to ApplicationMaster only when NodeManager is added/removed/'change
in NM state i.e Usable/Unusable'. Otherwise no updates are sent to AM  even at least one time
in application life time!!
> 
> I believe it is implemented purposefully to intimate AM about only unusable nodes. 
> 
> Thanks & Regards
> Rohith Sharma K S
> 
> -----Original Message-----
> From: Steve Loughran [mailto:stevel@hortonworks.com] 
> Sent: 18 October 2015 23:48
> To: yarn-dev@hadoop.apache.org
> Subject: When does the RM ever send out node map updates?
> 
> 
> I've been modifying the slider AM to wait for NM updates of node maps -but I'm not seeing
any onNodesUpdated() callbacks, at least for an AM which doesn't instantiate any containers.
> 
> When/how do those updates come?
> 

Mime
View raw message