hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6017) node manager physical memory leak
Date Fri, 23 Dec 2016 08:02:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15772254#comment-15772254
] 

Rajesh Balamohan commented on YARN-6017:
----------------------------------------

No. from JVM's accounting perspective it is still at 2048 as per the logs. But need to check
if it is anything to do with JVM's internal code itself or netty.  Have you tried with other
JDK versions?

> node manager physical memory leak
> ---------------------------------
>
>                 Key: YARN-6017
>                 URL: https://issues.apache.org/jira/browse/YARN-6017
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 2.7.1
>         Environment: OS:
> Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64
x86_64 GNU/Linux
> jvm:
> java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>            Reporter: chenrongwei
>         Attachments: 31169_smaps.txt, 31169_smaps.txt
>
>
> In our produce environment, node manager's jvm memory has been set to '-Xmx2048m',but
we notice that after a long time running the process' actual physical memory size had been
reached to 12g (we got this value by top command as follow).
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 31169 data      20   0 13.2g  12g 6092 S 16.9 13.0  49183:13 java
> 31169:   /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs
-Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs -Dhadoop.log.file=yarn-data-nodemanager.log
-Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data -Dhadoop.root.logger=INFO,RFA
-Dyarn.root.logger=INFO,RFA -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native
-Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M -XX:+UseC
> Address           Kbytes Mode  Offset           Device    Mapping
> 0000000000400000       4 r-x-- 0000000000000000 008:00001 java
> 0000000000600000       4 rw--- 0000000000000000 008:00001 java
> 0000000000601000 10094936 rw--- 0000000000000000 000:00000   [ anon ]
> 0000000770000000 2228224 rw--- 0000000000000000 000:00000   [ anon ]
> 00000007f8000000  131072 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325ee00000     128 r-x-- 0000000000000000 008:00001 ld-2.12.so
> 000000325f01f000       4 r---- 000000000001f000 008:00001 ld-2.12.so
> 000000325f020000       4 rw--- 0000000000020000 008:00001 ld-2.12.so
> 000000325f021000       4 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325f200000    1576 r-x-- 0000000000000000 008:00001 libc-2.12.so
> 000000325f38a000    2048 ----- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58a000      16 r---- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58e000       4 rw--- 000000000018e000 008:00001 libc-2.12.so
> 000000325f58f000      20 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325f600000      92 r-x-- 0000000000000000 008:00001 libpthread-2.12.so
> 000000325f617000    2048 ----- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f817000       4 r---- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f818000       4 rw--- 0000000000018000 008:00001 libpthread-2.12.so
> 000000325f819000      16 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325fa00000       8 r-x-- 0000000000000000 008:00001 libdl-2.12.so
> 000000325fa02000    2048 ----- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc02000       4 r---- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc03000       4 rw--- 0000000000003000 008:00001 libdl-2.12.so
> 000000325fe00000      28 r-x-- 0000000000000000 008:00001 librt-2.12.so
> 000000325fe07000    2044 ----- 0000000000007000 008:00001 librt-2.12.so
> 0000003260006000       4 r---- 0000000000006000 008:00001 librt-2.12.so
> 0000003260007000       4 rw--- 0000000000007000 008:00001 librt-2.12.so
> 0000003260200000     524 r-x-- 0000000000000000 008:00001 libm-2.12.so
> 0000003260283000    2044 ----- 0000000000083000 008:00001 libm-2.12.so
> 0000003260482000       4 r---- 0000000000082000 008:00001 libm-2.12.so
> 0000003260483000       4 rw--- 0000000000083000 008:00001 libm-2.12.so
> 0000003261200000      88 r-x-- 0000000000000000 008:00001 libresolv-2.12.so
> 0000003261216000    2048 ----- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261416000       4 r---- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261417000       4 rw--- 0000000000017000 008:00001 libresolv-2.12.so



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message