hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "karthik (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-15309) GZIP HFile Compression & Decompression Memory Leak, Similar Issue HBASE-5516
Date Thu, 03 Mar 2016 13:18:18 GMT

     [ https://issues.apache.org/jira/browse/HBASE-15309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

karthik resolved HBASE-15309.
-----------------------------
       Resolution: Not A Bug
     Hadoop Flags: Reviewed
    Fix Version/s: 0.94.28

Fix :  Hbase "hbase.regionserver.handler.count" shouldn't be exceeded then available CPU core.


> GZIP HFile Compression & Decompression Memory Leak, Similar Issue HBASE-5516
> ----------------------------------------------------------------------------
>
>                 Key: HBASE-15309
>                 URL: https://issues.apache.org/jira/browse/HBASE-15309
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile
>    Affects Versions: 0.94.6
>            Reporter: karthikp
>            Assignee: Todd Lipcon
>            Priority: Blocker
>             Fix For: 0.94.28
>
>
> I'd memory leak issue in regionserver process, VM and RSS memory continiously increasing
64MB.
> I'm using hbase-0.94.6 , MALLOC_ARENA_MAX=4 was set in hbase-env.sh
> I've huge write load and frequent minor compaction, We've used GZip hfile compression.
> Max java regionserver heap size is 32GB.
> {noformat}
> top - 14:28:30 up 201 days, 21:06,  3 users,  load average: 5.67, 3.72, 3.31
> Tasks: 803 total,   1 running, 802 sleeping,   0 stopped,   0 zombie
> Cpu(s):  8.3%us,  2.1%sy,  0.0%ni, 85.9%id,  3.5%wa,  0.0%hi,  0.1%si,  0.0%st
> Mem:  65932340k total, 63961912k used,  1970428k free,  2394528k buffers
> Swap: 29659132k total,    63532k used, 29595600k free,  1095268k cached
>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  57335 hbase     20   0 46.4g  44g 9296 S 98.2 70.9  13319:10 java
> {noformat}
> {noformat}
>  [hbase@xxxxxx-hslave ~]$  pmap -x 57335 | sort -k 3 -nr | more
> total kB        48695984 46765792 46756512
> 00007ff312460000 33171072 33169464 33169464 rwx--    [ anon ]
> 000000004010a000 1448552 1448552 1448552 rwx--    [ anon ]
> 00007ff2d1810000  612120  603124  603124 rwx--    [ anon ]
> 00007ff2fadff000  383364  383364  383364 rwx--    [ anon ]
> 00007ff0e8000000  131072  131072  131072 rwx--    [ anon ]
> 00007ff218000000  131048  131048  131048 rwx--    [ anon ]
> 00007ff128000000  131068  131048  131048 rwx--    [ anon ]
> 00007ff230000000  131044  131044  131044 rwx--    [ anon ]
> 00007ff000000000  131036  131036  131036 rwx--    [ anon ]
> 00007fefe0000000  131060  131036  131036 rwx--    [ anon ]
> 00007ff23c000000   65536   65536   65536 rwx--    [ anon ]
> 00007ff0a4000000   65536   65536   65536 rwx--    [ anon ]
> 00007ff054000000   65536   65536   65536 rwx--    [ anon ]
> 00007ff01c000000   65536   65536   65536 rwx--    [ anon ]
> 00007fefb4000000   65536   65536   65536 rwx--    [ anon ]
> 00007ff22c000000   65532   65532   65532 rwx--    [ anon ]
> 00007ff110000000   65532   65532   65532 rwx--    [ anon ]
> 00007ff10c000000   65532   65532   65532 rwx--    [ anon ]
> 00007ff0b8000000   65532   65532   65532 rwx--    [ anon ]
> 00007ff09c000000   65532   65532   65532 rwx--    [ anon ]
> 00007feff8000000   65532   65532   65532 rwx--    [ anon ]
> 00007ff250000000   65528   65528   65528 rwx--    [ anon ]
> --More--
> {noformat}
> {noformat}
> $pmap -x 57335 | awk '{print $3}' | awk '{ if($i<65536 && $i>64000) print
$i}' | wc -l
> 146
> {noformat}
> Regionserver process has many anon pages, size varying from 64000 to 65536 KB that shown
above.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message