hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6585) RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
Date Fri, 02 Jun 2017 17:05:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035055#comment-16035055

Eric Payne commented on YARN-6585:

bq. I don't understand why we needed to add a deprecated newInstance method
I'm also confused about the need for {{void setNodeLabelsList(Set<String> labels)}}
and {{Set<String> getNodeLabelsList()}} in {{ddToClusterNodeLabelsRequest\[PBImpl\]}}.
AFAICT, the code added to {{initLocalNodeLabels}} should be sufficient:
    if (this.updatedNodeLabels.isEmpty()) {
      List<String> deprecatedLabelsList = p.getDeprecatedNodeLabelsList();
      for (String l : deprecatedLabelsList) {

Also, I don't think the unit test is actually testing the above code. I took out the above
lines, ran the test, and it still succeeded.

In fact, I think this will be difficult to test, since {{AddToClusterNodeLabelsRequestPBImp#initLocalNodeLabelsl}}
is called by {{FileSystemNodeLabelsStore#loadFromMirror}}, which is reading the nodelabel.mirror
and nodelabel.editlog files from HDFS, and the current test doesn't seem to be mocking any
part of that. I'm still thinking about this one ;-/

> RM fails to start when upgrading from 2.7 to 2.8 for clusters with node labels.
> -------------------------------------------------------------------------------
>                 Key: YARN-6585
>                 URL: https://issues.apache.org/jira/browse/YARN-6585
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Eric Payne
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-6585.0001.patch, YARN-6585.0002.patch
> {noformat}
> Caused by: java.io.IOException: Not all labels being replaced contained by known label
collections, please check, new labels=[abc]
>         at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.checkReplaceLabelsOnNode(CommonNodeLabelsManager.java:718)
>         at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.replaceLabelsOnNode(CommonNodeLabelsManager.java:737)
>         at org.apache.hadoop.yarn.server.resourcemanager.nodelabels.RMNodeLabelsManager.replaceLabelsOnNode(RMNodeLabelsManager.java:189)
>         at org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.loadFromMirror(FileSystemNodeLabelsStore.java:181)
>         at org.apache.hadoop.yarn.nodelabels.FileSystemNodeLabelsStore.recover(FileSystemNodeLabelsStore.java:208)
>         at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.initNodeLabelStore(CommonNodeLabelsManager.java:251)
>         at org.apache.hadoop.yarn.nodelabels.CommonNodeLabelsManager.serviceStart(CommonNodeLabelsManager.java:265)
>         at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>         ... 13 more
> {noformat}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message