hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chenrongwei (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6017) node manager physical memory leak
Date Fri, 23 Dec 2016 05:59:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-6017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15772031#comment-15772031
] 

chenrongwei commented on YARN-6017:
-----------------------------------

$ jmap -heap 31169
Attaching to process ID 31169, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 24.65-b04

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 2147483648 (2048.0MB)
   NewSize          = 805306368 (768.0MB)
   MaxNewSize       = 805306368 (768.0MB)
   OldSize          = 5439488 (5.1875MB)
   NewRatio         = 2
   SurvivorRatio    = 8
   PermSize         = 134217728 (128.0MB)
   MaxPermSize      = 268435456 (256.0MB)
   G1HeapRegionSize = 0 (0.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 724828160 (691.25MB)
   used     = 220609280 (210.389404296875MB)
   free     = 504218880 (480.860595703125MB)
   30.43608018761302% used
Eden Space:
   capacity = 644349952 (614.5MB)
   used     = 197290320 (188.1507110595703MB)
   free     = 447059632 (426.3492889404297MB)
   30.61850464761112% used
>From Space:
   capacity = 80478208 (76.75MB)
   used     = 23318960 (22.238693237304688MB)
   free     = 57159248 (54.51130676269531MB)
   28.97549607466409% used
To Space:
   capacity = 80478208 (76.75MB)
   used     = 0 (0.0MB)
   free     = 80478208 (76.75MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 1342177280 (1280.0MB)
   used     = 596879392 (569.2285461425781MB)
   free     = 745297888 (710.7714538574219MB)
   44.470980167388916% used
Perm Generation:
   capacity = 134217728 (128.0MB)
   used     = 48817648 (46.55613708496094MB)
   free     = 85400080 (81.44386291503906MB)
   36.37198209762573% used

15971 interned Strings occupying 1520304 bytes.

> node manager physical memory leak
> ---------------------------------
>
>                 Key: YARN-6017
>                 URL: https://issues.apache.org/jira/browse/YARN-6017
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: yarn
>    Affects Versions: 2.7.1
>         Environment: OS:
> Linux guomai124041 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64
x86_64 GNU/Linux
> jvm:
> java version "1.7.0_65"
> Java(TM) SE Runtime Environment (build 1.7.0_65-b17)
> Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)
>            Reporter: chenrongwei
>
> In our produce environment, node manager's jvm memory has been set to '-Xmx2048m',but
we notice that after a long time running the process' actual physical memory size had been
reached to 12g (we got this value by top command as follow).
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 31169 data      20   0 13.2g  12g 6092 S 16.9 13.0  49183:13 java
> 31169:   /usr/local/jdk/bin/java -Dproc_nodemanager -Xmx2048m -Dhadoop.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs
-Dyarn.log.dir=/home/data/programs/apache-hadoop-2.7.1/logs -Dhadoop.log.file=yarn-data-nodemanager.log
-Dyarn.log.file=yarn-data-nodemanager.log -Dyarn.home.dir= -Dyarn.id.str=data -Dhadoop.root.logger=INFO,RFA
-Dyarn.root.logger=INFO,RFA -Djava.library.path=/home/data/programs/apache-hadoop-2.7.1/lib/native
-Dyarn.policy.file=hadoop-policy.xml -XX:PermSize=128M -XX:MaxPermSize=256M -XX:+UseC
> Address           Kbytes Mode  Offset           Device    Mapping
> 0000000000400000       4 r-x-- 0000000000000000 008:00001 java
> 0000000000600000       4 rw--- 0000000000000000 008:00001 java
> 0000000000601000 10094936 rw--- 0000000000000000 000:00000   [ anon ]
> 0000000770000000 2228224 rw--- 0000000000000000 000:00000   [ anon ]
> 00000007f8000000  131072 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325ee00000     128 r-x-- 0000000000000000 008:00001 ld-2.12.so
> 000000325f01f000       4 r---- 000000000001f000 008:00001 ld-2.12.so
> 000000325f020000       4 rw--- 0000000000020000 008:00001 ld-2.12.so
> 000000325f021000       4 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325f200000    1576 r-x-- 0000000000000000 008:00001 libc-2.12.so
> 000000325f38a000    2048 ----- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58a000      16 r---- 000000000018a000 008:00001 libc-2.12.so
> 000000325f58e000       4 rw--- 000000000018e000 008:00001 libc-2.12.so
> 000000325f58f000      20 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325f600000      92 r-x-- 0000000000000000 008:00001 libpthread-2.12.so
> 000000325f617000    2048 ----- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f817000       4 r---- 0000000000017000 008:00001 libpthread-2.12.so
> 000000325f818000       4 rw--- 0000000000018000 008:00001 libpthread-2.12.so
> 000000325f819000      16 rw--- 0000000000000000 000:00000   [ anon ]
> 000000325fa00000       8 r-x-- 0000000000000000 008:00001 libdl-2.12.so
> 000000325fa02000    2048 ----- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc02000       4 r---- 0000000000002000 008:00001 libdl-2.12.so
> 000000325fc03000       4 rw--- 0000000000003000 008:00001 libdl-2.12.so
> 000000325fe00000      28 r-x-- 0000000000000000 008:00001 librt-2.12.so
> 000000325fe07000    2044 ----- 0000000000007000 008:00001 librt-2.12.so
> 0000003260006000       4 r---- 0000000000006000 008:00001 librt-2.12.so
> 0000003260007000       4 rw--- 0000000000007000 008:00001 librt-2.12.so
> 0000003260200000     524 r-x-- 0000000000000000 008:00001 libm-2.12.so
> 0000003260283000    2044 ----- 0000000000083000 008:00001 libm-2.12.so
> 0000003260482000       4 r---- 0000000000082000 008:00001 libm-2.12.so
> 0000003260483000       4 rw--- 0000000000083000 008:00001 libm-2.12.so
> 0000003261200000      88 r-x-- 0000000000000000 008:00001 libresolv-2.12.so
> 0000003261216000    2048 ----- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261416000       4 r---- 0000000000016000 008:00001 libresolv-2.12.so
> 0000003261417000       4 rw--- 0000000000017000 008:00001 libresolv-2.12.so



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message