hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HBASE-5516) GZip leading to memory leak in 0.90. Fix similar to HBASE-5387 needed for 0.90.
Date Thu, 08 Mar 2012 13:43:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225202#comment-13225202
] 

ramkrishna.s.vasudevan edited comment on HBASE-5516 at 3/8/12 1:42 PM:
-----------------------------------------------------------------------

{code}
top - 08:41:24 up 28 days, 17:46,  4 users,  load average: 3.23, 2.73, 2.52
Tasks: 308 total,   1 running, 307 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.3%us,  2.4%sy,  0.0%ni, 78.8%id,  7.8%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:     48264M total,    48125M used,      139M free,     4370M buffers
Swap:    51199M total,       53M used,    51146M free,    20926M cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                                                        
30934 root      20   0 23.2g  20g  12m S  144 42.6 314:25.74 java                        
                                                                                         
30717 root      20   0 4859m 200m  10m S   50  0.4  66:09.54 java                        
                                                                                         
 7351 root      20   0     0    0    0 D    2  0.0  14:30.49 kjournald                   
                                                                                         
   38 root      20   0     0    0    0 S    1  0.0   0:36.31 events/3                    
                                                                                         
  127 root      20   0     0    0    0 S    1  0.0  48:30.54 kswapd0                     
                                                                                         
  128 root      20   0     0    0    0 S    1  0.0  47:48.75 kswapd1                     
                                                                                         
 4877 root      20   0  8968  400  260 S    1  0.0   8:48.69 irqbalance                  
                                                                                         
12644 root      20   0  8900 1356  852 R    1  0.0   0:00.91 top                  

====================================================

{code}

The above reports are before patch.  You can see the native memory going beyond 20g.

{code}
top - 13:41:13 up 28 days, 22:46,  4 users,  load average: 0.00, 0.02, 0.00
Tasks: 298 total,   1 running, 297 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni, 99.9%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:     48264M total,    27960M used,    20304M free,     4861M buffers
Swap:    51199M total,       53M used,    51146M free,    12556M cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                                                        
 6822 root      20   0  382m 7916 4116 S    0  0.0  31:22.40 nautilus                    
                                                                                         
13197 root      20   0 4828m 189m  10m S    0  0.4  60:49.90 java                        
                                                                                         
13424 root      20   0 8901m 8.1g  12m S    0 17.2 407:56.80 java                        
                                                                                         
27929 root      20   0  3620 1072  600 S    0  0.0   0:05.99 nmon                        
                                                                                         
28225 root      20   0  8900 1344  856 R    0  0.0   0:00.04 top                         
                                                                                         
    1 root      20   0 10376  340  304 S    0  0.0   0:13.82 init                        
                                                                                         
    2 root      20   0     0    0    0 S    0  0.0   0:00.25 kthreadd                    
                                                                                         
    3 root      RT   0     0    0    0 S    0  0.0   0:01.22 migration/0                 
                                                                                         
    4 root      20   0     0    0    0 S    0  0.0   2:45.12 ksoftirqd/0                 
                                                                                         
    5 root      RT   0     0    0    0 S    0  0.0   0:00.66 migration/1                 
                                                                                         
    6 root      20   0     0    0    0 S    0  0.0   0:22.16 ksoftirqd/1 
{code}
In the above the process with 8.1 g is the RS process and we can see there is no increase.

The test was done over a period of 3 hours with write load.
                
      was (Author: ram_krish):
    {code}
top - 08:41:24 up 28 days, 17:46,  4 users,  load average: 3.23, 2.73, 2.52
Tasks: 308 total,   1 running, 307 sleeping,   0 stopped,   0 zombie
Cpu(s): 10.3%us,  2.4%sy,  0.0%ni, 78.8%id,  7.8%wa,  0.0%hi,  0.7%si,  0.0%st
Mem:     48264M total,    48125M used,      139M free,     4370M buffers
Swap:    51199M total,       53M used,    51146M free,    20926M cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                     
                                                                                        
30934 root      20   0 23.2g  20g  12m S  144 42.6 314:25.74 java                        
                                                                                         
30717 root      20   0 4859m 200m  10m S   50  0.4  66:09.54 java                        
                                                                                         
 7351 root      20   0     0    0    0 D    2  0.0  14:30.49 kjournald                   
                                                                                         
   38 root      20   0     0    0    0 S    1  0.0   0:36.31 events/3                    
                                                                                         
  127 root      20   0     0    0    0 S    1  0.0  48:30.54 kswapd0                     
                                                                                         
  128 root      20   0     0    0    0 S    1  0.0  47:48.75 kswapd1                     
                                                                                         
 4877 root      20   0  8968  400  260 S    1  0.0   8:48.69 irqbalance                  
                                                                                         
12644 root      20   0  8900 1356  852 R    1  0.0   0:00.91 top                  

====================================================

{code}

The above reports are before patch.  You can see the native memory going beyond 20g.

{code}
{code}
                  
> GZip leading to memory leak in 0.90.  Fix similar to HBASE-5387 needed for 0.90.
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-5516
>                 URL: https://issues.apache.org/jira/browse/HBASE-5516
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.90.5
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 0.90.7
>
>         Attachments: HBASE-5516_2_0.90.patch
>
>
> Usage of GZip is leading to resident memory leak in 0.90.
> We need to have something similar to HBASE-5387 in 0.90. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message