reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergiy Matusevych (JIRA)" <>
Subject [jira] [Commented] (REEF-1669) Removing YarnClient from YarnContainerManager
Date Wed, 16 Nov 2016 15:19:58 GMT


Sergiy Matusevych commented on REEF-1669:

I am not 100% sure yet, but I *might* need {{YarnClient}} on the driver side for the REEF-on-REEF
or REEF-on-Spark scenario. Most likely, it will not be a part of the container manager, but
as a component of the {{REEFEnvironment}}. I'll figure it out and let you know - but please
do not delete it just yet, or do not merge the PR if you already have it.

> Removing YarnClient from YarnContainerManager
> ---------------------------------------------
>                 Key: REEF-1669
>                 URL:
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF
>            Reporter: Julia
>            Assignee: Julia
> Currently we have both YarnClient and resourceManager (AMRMClientAsync) in YarnContainerManager.
We call init() and start() for both of them. But inside driver, we are not able to get correct
YarnConfiguration for YarnClient, that makes yarnClient.getNodeReports() fail to connect to
> In YarnContainerManager, we already have onNodesUpdated() which gives the list of the
NodeReports on the fly.  yarnClient.getNodeReports() was called at beginning and it only gives
a static list. So looks like we don't need to call it anyway. 
> Based on RM folks, REEF should not use YarnClient at driver side. So the proposal it
to to remove YarnClient from YarnContainerManager. 

This message was sent by Atlassian JIRA

View raw message