hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Yu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10642) Provide option to limit heap memory consumed by dynamic metrics2 metrics
Date Sun, 18 Oct 2015 16:18:05 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Yu updated HADOOP-10642:
----------------------------
    Description: 
User sunweiei provided the following jmap output in HBase 0.96 deployment:
{code}
 num     #instances         #bytes  class name
----------------------------------------------
   1:      14917882     3396492464  [C
   2:       1996994     2118021808  [B
   3:      43341650     1733666000  java.util.LinkedHashMap$Entry
   4:      14453983     1156550896  [Ljava.util.HashMap$Entry;
   5:      14446577      924580928  
org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2
{code}
Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls
to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java.

This scenario would arise when large number of regions are tracked through metrics2 dynamically.
Interns class doesn't provide API to remove entries in its internal Map.

One solution is to provide an option that allows skipping calls to Interns.info() in metrics2/lib/MetricsRegistry.java

  was:
User sunweiei provided the following jmap output in HBase 0.96 deployment:
{code}
 num     #instances         #bytes  class name
----------------------------------------------
   1:      14917882     3396492464  [C
   2:       1996994     2118021808  [B
   3:      43341650     1733666000  java.util.LinkedHashMap$Entry
   4:      14453983     1156550896  [Ljava.util.HashMap$Entry;
   5:      14446577      924580928  
org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2
{code}
Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls
to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java.
This scenario would arise when large number of regions are tracked through metrics2 dynamically.
Interns class doesn't provide API to remove entries in its internal Map.

One solution is to provide an option that allows skipping calls to Interns.info() in metrics2/lib/MetricsRegistry.java


> Provide option to limit heap memory consumed by dynamic metrics2 metrics
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-10642
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10642
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Ted Yu
>
> User sunweiei provided the following jmap output in HBase 0.96 deployment:
> {code}
>  num     #instances         #bytes  class name
> ----------------------------------------------
>    1:      14917882     3396492464  [C
>    2:       1996994     2118021808  [B
>    3:      43341650     1733666000  java.util.LinkedHashMap$Entry
>    4:      14453983     1156550896  [Ljava.util.HashMap$Entry;
>    5:      14446577      924580928  
> org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2
> {code}
> Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls
to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java.
> This scenario would arise when large number of regions are tracked through metrics2 dynamically.
> Interns class doesn't provide API to remove entries in its internal Map.
> One solution is to provide an option that allows skipping calls to Interns.info() in
metrics2/lib/MetricsRegistry.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message