Return-Path: X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D108A184F3 for ; Sun, 18 Oct 2015 16:18:05 +0000 (UTC) Received: (qmail 40128 invoked by uid 500); 18 Oct 2015 16:18:05 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 40079 invoked by uid 500); 18 Oct 2015 16:18:05 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 40064 invoked by uid 99); 18 Oct 2015 16:18:05 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 18 Oct 2015 16:18:05 +0000 Date: Sun, 18 Oct 2015 16:18:05 +0000 (UTC) From: "Ted Yu (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HADOOP-10642) Provide option to limit heap memory consumed by dynamic metrics2 metrics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-10642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Yu updated HADOOP-10642: ---------------------------- Description: User sunweiei provided the following jmap output in HBase 0.96 deployment: {code} num #instances #bytes class name ---------------------------------------------- 1: 14917882 3396492464 [C 2: 1996994 2118021808 [B 3: 43341650 1733666000 java.util.LinkedHashMap$Entry 4: 14453983 1156550896 [Ljava.util.HashMap$Entry; 5: 14446577 924580928 org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2 {code} Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java. This scenario would arise when large number of regions are tracked through metrics2 dynamically. Interns class doesn't provide API to remove entries in its internal Map. One solution is to provide an option that allows skipping calls to Interns.info() in metrics2/lib/MetricsRegistry.java was: User sunweiei provided the following jmap output in HBase 0.96 deployment: {code} num #instances #bytes class name ---------------------------------------------- 1: 14917882 3396492464 [C 2: 1996994 2118021808 [B 3: 43341650 1733666000 java.util.LinkedHashMap$Entry 4: 14453983 1156550896 [Ljava.util.HashMap$Entry; 5: 14446577 924580928 org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2 {code} Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java. This scenario would arise when large number of regions are tracked through metrics2 dynamically. Interns class doesn't provide API to remove entries in its internal Map. One solution is to provide an option that allows skipping calls to Interns.info() in metrics2/lib/MetricsRegistry.java > Provide option to limit heap memory consumed by dynamic metrics2 metrics > ------------------------------------------------------------------------ > > Key: HADOOP-10642 > URL: https://issues.apache.org/jira/browse/HADOOP-10642 > Project: Hadoop Common > Issue Type: Improvement > Components: metrics > Reporter: Ted Yu > > User sunweiei provided the following jmap output in HBase 0.96 deployment: > {code} > num #instances #bytes class name > ---------------------------------------------- > 1: 14917882 3396492464 [C > 2: 1996994 2118021808 [B > 3: 43341650 1733666000 java.util.LinkedHashMap$Entry > 4: 14453983 1156550896 [Ljava.util.HashMap$Entry; > 5: 14446577 924580928 > org.apache.hadoop.metrics2.lib.Interns$CacheWith2Keys$2 > {code} > Heap consumption by Interns$CacheWith2Keys$2 (and indirectly by [C) could be due to calls to Interns.info() in DynamicMetricsRegistry which was cloned off metrics2/lib/MetricsRegistry.java. > This scenario would arise when large number of regions are tracked through metrics2 dynamically. > Interns class doesn't provide API to remove entries in its internal Map. > One solution is to provide an option that allows skipping calls to Interns.info() in metrics2/lib/MetricsRegistry.java -- This message was sent by Atlassian JIRA (v6.3.4#6332)