hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "karthikp (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-15309) GZIP HFile Compression & Decompression Memory Leak, Similar Issue HBASE-5516
Date Tue, 23 Feb 2016 06:14:18 GMT
karthikp created HBASE-15309:
--------------------------------

             Summary: GZIP HFile Compression & Decompression Memory Leak, Similar Issue
HBASE-5516
                 Key: HBASE-15309
                 URL: https://issues.apache.org/jira/browse/HBASE-15309
             Project: HBase
          Issue Type: Bug
          Components: HFile
    Affects Versions: 0.94.6
            Reporter: karthikp
            Priority: Critical


I'd memory leak issue in regionserver process, VM and RSS memory continiously increasing 64MB.
I'm using hbase-0.94.6 , MALLOC_ARENA_MAX=4 was set in hbase-env.sh
I've huge write load and frequent minor compaction, We've used GZip hfile compression.
Max java regionserver heap size is 32GB.


{noformat}
top - 14:28:30 up 201 days, 21:06,  3 users,  load average: 5.67, 3.72, 3.31
Tasks: 803 total,   1 running, 802 sleeping,   0 stopped,   0 zombie
Cpu(s):  8.3%us,  2.1%sy,  0.0%ni, 85.9%id,  3.5%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:  65932340k total, 63961912k used,  1970428k free,  2394528k buffers
Swap: 29659132k total,    63532k used, 29595600k free,  1095268k cached

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 57335 hbase     20   0 46.4g  44g 9296 S 98.2 70.9  13319:10 java
{noformat}

{noformat}
 [hbase@xxxxxx-hslave ~]$  pmap -x 57335 | sort -k 3 -nr | more
total kB        48695984 46765792 46756512
00007ff312460000 33171072 33169464 33169464 rwx--    [ anon ]
000000004010a000 1448552 1448552 1448552 rwx--    [ anon ]
00007ff2d1810000  612120  603124  603124 rwx--    [ anon ]
00007ff2fadff000  383364  383364  383364 rwx--    [ anon ]
00007ff0e8000000  131072  131072  131072 rwx--    [ anon ]
00007ff218000000  131048  131048  131048 rwx--    [ anon ]
00007ff128000000  131068  131048  131048 rwx--    [ anon ]
00007ff230000000  131044  131044  131044 rwx--    [ anon ]
00007ff000000000  131036  131036  131036 rwx--    [ anon ]
00007fefe0000000  131060  131036  131036 rwx--    [ anon ]
00007ff23c000000   65536   65536   65536 rwx--    [ anon ]
00007ff0a4000000   65536   65536   65536 rwx--    [ anon ]
00007ff054000000   65536   65536   65536 rwx--    [ anon ]
00007ff01c000000   65536   65536   65536 rwx--    [ anon ]
00007fefb4000000   65536   65536   65536 rwx--    [ anon ]
00007ff22c000000   65532   65532   65532 rwx--    [ anon ]
00007ff110000000   65532   65532   65532 rwx--    [ anon ]
00007ff10c000000   65532   65532   65532 rwx--    [ anon ]
00007ff0b8000000   65532   65532   65532 rwx--    [ anon ]
00007ff09c000000   65532   65532   65532 rwx--    [ anon ]
00007feff8000000   65532   65532   65532 rwx--    [ anon ]
00007ff250000000   65528   65528   65528 rwx--    [ anon ]
--More--
{noformat}

{noformat}
$pmap -x 57335 | awk '{print $3}' | awk '{ if($i<65536 && $i>64000) print $i}'
| wc -l
146
{noformat}

Will MALLOC_AREANA have exactly 65536KB? 
Regionserver process has many anon pages, size varying from 64000 to 65536 KB that shown above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message