hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3445) Cache runningApps in RMNode for getting running apps on given NodeId
Date Tue, 30 Jun 2015 01:00:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606766#comment-14606766
] 

Ming Ma commented on YARN-3445:
-------------------------------

Thanks [~djp]. Quick questions:

* Regarding the extra memory this might take. Say you have 10K NM, 1000 apps per NM ( an application
will remain in a RMNodeImpl list even though there are no more containers running on that
NM, but the app still run on another NM.). That is around 10M entries. So it should be ok
for RM.
* Do you need synchronizedList in the following list? It looks like the access of runningApplications
are protected by RMNodeImpl's readLock and writeLock.
{noformat}
private final List<ApplicationId> runningApplications =
    Collections.synchronizedList(new ArrayList<ApplicationId>());
{noromat}

> Cache runningApps in RMNode for getting running apps on given NodeId
> --------------------------------------------------------------------
>
>                 Key: YARN-3445
>                 URL: https://issues.apache.org/jira/browse/YARN-3445
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Junping Du
>            Assignee: Junping Du
>         Attachments: YARN-3445-v2.patch, YARN-3445-v3.1.patch, YARN-3445-v3.patch, YARN-3445.patch
>
>
> Per discussion in YARN-3334, we need filter out unnecessary collectors info from RM in
heartbeat response. Our propose is to add cache for runningApps in RMNode, so RM only send
collectors for local running apps back. This is also needed in YARN-914 (graceful decommission)
that if no running apps in NM which is in decommissioning stage, it will get decommissioned
immediately. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message