hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3990) AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected
Date Thu, 30 Jul 2015 14:58:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647743#comment-14647743
] 

Jason Lowe commented on YARN-3990:
----------------------------------

Please look into the java and release audit warnings.

The new TestNMUpdateEvent file probably should just be named TestNodesListManager so we can
add more tests specific to the NodesListManager there.

Rather than using mock RM and NM objects and running some RMApps it seems like we could have
just isolated the NodesListManager more directly.  We can hand it a mocked RMContext that
returns a pre-baked list of apps that are alive and apps that are finished and then verify
when we send a node usable/unusable event that the appropriate update events are sent to the
appropriate apps by registering our own event handler with something like a drain or inline
dispatcher.  Not a must fix, just wondering if that approach was considered.

> AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when Node is connected/disconnected
> --------------------------------------------------------------------------------------------
>
>                 Key: YARN-3990
>                 URL: https://issues.apache.org/jira/browse/YARN-3990
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Rohith Sharma K S
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: 0001-YARN-3990.patch, 0002-YARN-3990.patch
>
>
> Whenever node is added or removed, NodeListManager sends RMAppNodeUpdateEvent to all
the applications that are in the rmcontext. But for finished/killed/failed applications it
is not required to send these events. Additional check for wheather app is finished/killed/failed
would minimizes the unnecessary events
> {code}
>   public void handle(NodesListManagerEvent event) {
>     RMNode eventNode = event.getNode();
>     switch (event.getType()) {
>     case NODE_UNUSABLE:
>       LOG.debug(eventNode + " reported unusable");
>       unusableRMNodesConcurrentSet.add(eventNode);
>       for(RMApp app: rmContext.getRMApps().values()) {
>         this.rmContext
>             .getDispatcher()
>             .getEventHandler()
>             .handle(
>                 new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
>                     RMAppNodeUpdateType.NODE_UNUSABLE));
>       }
>       break;
>     case NODE_USABLE:
>       if (unusableRMNodesConcurrentSet.contains(eventNode)) {
>         LOG.debug(eventNode + " reported usable");
>         unusableRMNodesConcurrentSet.remove(eventNode);
>       }
>       for (RMApp app : rmContext.getRMApps().values()) {
>         this.rmContext
>             .getDispatcher()
>             .getEventHandler()
>             .handle(
>                 new RMAppNodeUpdateEvent(app.getApplicationId(), eventNode,
>                     RMAppNodeUpdateType.NODE_USABLE));
>       }
>       break;
>     default:
>       LOG.error("Ignoring invalid eventtype " + event.getType());
>     }
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message