hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-6523) RM requires large memory in sending out security tokens as part of Node Heartbeat in large cluster
Date Tue, 25 Apr 2017 02:14:04 GMT
Naganarasimha G R created YARN-6523:
---------------------------------------

             Summary: RM requires large memory in sending out security tokens as part of Node
Heartbeat in large cluster
                 Key: YARN-6523
                 URL: https://issues.apache.org/jira/browse/YARN-6523
             Project: Hadoop YARN
          Issue Type: Bug
          Components: RM
    Affects Versions: 2.7.3, 2.8.0
            Reporter: Naganarasimha G R
            Assignee: Naganarasimha G R
            Priority: Critical


Currently as part of heartbeat response RM sets all application's tokens though all applications
might not be active on the node. On top of it NodeHeartbeatResponsePBImpl converts tokens
for each app into SystemCredentialsForAppsProto. Hence for each node and each heartbeat too
many SystemCredentialsForAppsProto objects were getting created.
We hit a OOM while testing for 2000 concurrent apps on 500 nodes cluster with 8GB RAM configured
for RM




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message