hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ajay Jadhav (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6188) Fix OOM issue with decommissioningNodesWatcher in the case of clusters with large number of nodes
Date Tue, 14 Feb 2017 20:26:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866576#comment-15866576
] 

Ajay Jadhav commented on YARN-6188:
-----------------------------------

It is hard to provide a unit test for OOM issue.

I have tested this by creating 4 large clusters (2000+ nodes) and execute a long running hive
and spark (on yarn) job.
While the jobs are running, resize the cluster to reduce the number of nodes.
The resource manager didn't restart and no OOM exception was seen in the logs.

> Fix OOM issue with decommissioningNodesWatcher in the case of clusters with large number
of nodes
> -------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6188
>                 URL: https://issues.apache.org/jira/browse/YARN-6188
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.9.0
>            Reporter: Ajay Jadhav
>             Fix For: 2.9.0, 3.0.0-alpha1
>
>         Attachments: YARN-6188.001.patch, YARN-6188.002.patch
>
>
> LogDecommissioningNodesStatus method in DecommissioningNodesWatcher uses StringBuilder
to append status of all
> decommissioning nodes for logging purpose.
> In the case of large number of decommissioning nodes, this leads to OOM exception. The
fix scopes StringBuilder so that in case of memory pressure, GC can kick in and free up the
memory.
> This is supposed to fix a bug introduced in https://issues.apache.org/jira/browse/YARN-4676



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message