hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1506) Replace set resource change on RMNode/SchedulerNode directly with event notification.
Date Fri, 14 Feb 2014 03:33:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901081#comment-13901081

Junping Du commented on YARN-1506:

Agree with Arun that this is not a blocker.

Hi [~bikassaha], Thanks for you review and comments. Sorry for replying late as just come
back from a long vacation. Please see my reply below:
bq. ADMIN_RESOURCE_UPDATE instead of RESOURCE_UPDATE for the enum would help clarify that
its a forced admin update. 
Ok. Will update it.

bq. Why not update the total capability here also (like we do for non-running node). When
the node reports back as healthy then we would probably need the new resource value, right?
For node that unusable (unhealthy, LOST or decommissioned), I think it may be simpler to just
log and warn rather than do any valid change. Or user may get confused that the node is still
usable. Thoughts? 

bq. Why are we doing this indirect subtraction via delta instead of simply clusterResource-=old;
clusterResource+=new. Its the same number of operations and less confusing to read.
Good point. Will update it.

bq. I think its crucial to have a more complete test (maybe using mockRM) that verifies the
flow from admin service to the scheduler. Most interesting would be the case when the node
is full allocated and then an update reduces the capacity. Thus resulting in -ve value of
available resource on the node. I am wary that this case may have bugs in handling the -ve
value in existing scheduler code because its unexpected. Its fine for the test to use the
default scheduler.
Agree. Although I am pretty sure it works fine so far from my offline integration test, we
have to add unit test to cover resource over-commitment case so any changes in future won't
break these assumptions. 

Will update patch soon.

> Replace set resource change on RMNode/SchedulerNode directly with event notification.
> -------------------------------------------------------------------------------------
>                 Key: YARN-1506
>                 URL: https://issues.apache.org/jira/browse/YARN-1506
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, scheduler
>            Reporter: Junping Du
>            Assignee: Junping Du
>            Priority: Blocker
>         Attachments: YARN-1506-v1.patch, YARN-1506-v2.patch, YARN-1506-v3.patch, YARN-1506-v4.patch,
YARN-1506-v5.patch, YARN-1506-v6.patch
> According to Vinod's comments on YARN-312 (https://issues.apache.org/jira/browse/YARN-312?focusedCommentId=13846087&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13846087),
we should replace RMNode.setResourceOption() with some resource change event.

This message was sent by Atlassian JIRA

View raw message