hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2923) Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration Setup
Date Thu, 30 Jul 2015 20:19:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-2923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648245#comment-14648245

Wangda Tan commented on YARN-2923:

Thanks for updating, [~Naganarasimha]! Some comments:

1) All script provider related configurations/logics should be removed. They should come with
different patch.
  Such as: {{case YarnConfiguration.SCRIPT_NODE_LABELS_PROVIDER:}}
  And {{public static final String SCRIPT_NODE_LABELS_PROVIDER = "script";}}
  Should be removed

2) For logics in NodeStatusUpdater. Things in my mind are:
- PreviousNodeLabels will be reset everytime if we do a fetch. (To avoid handle same node
labels as much as possible)
- Don't reset node label if fetched node label is incorrect. (This should be a part of error
handling, we should treat it's a error to be avoided instead of force reset it)
- Don't do check if new fetched node label is as same as previousNodeLabels. (Also, avoid
handle same node label)

A little cosmetic suggestion. I found {{startStatusUpdater}} is too complex, full of try catch,
etc. I suggest to make label related logic to be: a. Fetch label, check. b. handle response
from RM and post process. Each of them should a separated method to improve readability.

I suggest to keep provider within nodemanager (instead of yarn-server-common) for this patch,
we can move it if we decide to do that in the future.

Please let me know about your thoughts.


> Support configuration based NodeLabelsProvider Service in Distributed Node Label Configuration
> -----------------------------------------------------------------------------------------------------
>                 Key: YARN-2923
>                 URL: https://issues.apache.org/jira/browse/YARN-2923
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>             Fix For: 2.8.0
>         Attachments: YARN-2923.20141204-1.patch, YARN-2923.20141210-1.patch, YARN-2923.20150328-1.patch,
YARN-2923.20150404-1.patch, YARN-2923.20150517-1.patch
> As part of Distributed Node Labels configuration we need to support Node labels to be
configured in Yarn-site.xml. And on modification of Node Labels configuration in yarn-site.xml,
NM should be able to get modified Node labels from this NodeLabelsprovider service without
NM restart

This message was sent by Atlassian JIRA

View raw message