reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia (JIRA)" <>
Subject [jira] [Commented] (REEF-1669) Removing YarnClient from YarnContainerManager
Date Wed, 16 Nov 2016 19:25:58 GMT


Julia commented on REEF-1669:

I will only remove it from YarnContainerManager where is is mainly used to getNodeReports
which is not needed. 
I will send PR for review in a minute. As it blocks the REEF IMRU FT, we might have to merge
the change. But if this is going to cause issue or still needed somewhere else, we need to
figure out. 

> Removing YarnClient from YarnContainerManager
> ---------------------------------------------
>                 Key: REEF-1669
>                 URL:
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF
>            Reporter: Julia
>            Assignee: Julia
> Currently we have both YarnClient and resourceManager (AMRMClientAsync) in YarnContainerManager.
We call init() and start() for both of them. But inside driver, we are not able to get correct
YarnConfiguration for YarnClient, that makes yarnClient.getNodeReports() fail to connect to
> In YarnContainerManager, we already have onNodesUpdated() which gives the list of the
NodeReports on the fly.  yarnClient.getNodeReports() was called at beginning and it only gives
a static list. So looks like we don't need to call it anyway. 
> Based on RM folks, REEF should not use YarnClient at driver side. So the proposal it
to to remove YarnClient from YarnContainerManager. 

This message was sent by Atlassian JIRA

View raw message