hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15742) Reduce allocation of objects in metrics
Date Tue, 03 May 2016 10:06:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15268452#comment-15268452
] 

Hadoop QA commented on HBASE-15742:
-----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m 0s {color} |
{color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green}
The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red}
The patch doesn't appear to include any new or modified tests. Please justify why no new tests
are needed for this patch. Also please list what manual steps were performed to verify this
patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 34s {color}
| {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s {color} |
{color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 11s {color} |
{color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 7s {color}
| {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 9s {color}
| {color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 27s {color} |
{color:green} branch-1.2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s {color} |
{color:green} branch-1.2 passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s {color} |
{color:green} branch-1.2 passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 9s {color} | {color:red}
hbase-hadoop2-compat in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 19s {color} | {color:red}
hbase-hadoop2-compat in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s {color} | {color:red}
hbase-hadoop2-compat in the patch failed with JDK v1.8.0. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m 9s {color} | {color:red}
hbase-hadoop2-compat in the patch failed with JDK v1.7.0_79. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 9s {color} | {color:red}
hbase-hadoop2-compat in the patch failed with JDK v1.7.0_79. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 7s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 8s {color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color}
| {color:green} Patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 0m 37s {color} | {color:red}
Patch causes 30 errors with Hadoop v2.4.1. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 12s {color} | {color:red}
Patch causes 30 errors with Hadoop v2.5.2. {color} |
| {color:red}-1{color} | {color:red} hadoopcheck {color} | {color:red} 1m 47s {color} | {color:red}
Patch causes 30 errors with Hadoop v2.6.0. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 10s {color} | {color:red}
hbase-hadoop2-compat in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s {color} |
{color:green} the patch passed with JDK v1.8.0 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 10s {color} |
{color:green} the patch passed with JDK v1.7.0_79 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 9s {color} | {color:red}
hbase-hadoop2-compat in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 8s {color}
| {color:green} Patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 7m 13s {color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12801919/HBASE-15742-branch-1.2-v1.patch
|
| JIRA Issue | HBASE-15742 |
| Optional Tests |  asflicense  javac  javadoc  unit  findbugs  hadoopcheck  hbaseanti  checkstyle
 compile  |
| uname | Linux pietas.apache.org 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/test_framework/yetus-0.2.1/lib/precommit/personality/hbase.sh
|
| git revision | branch-1.2 / bf13941 |
| Default Java | 1.7.0_79 |
| Multi-JDK versions |  /home/jenkins/tools/java/jdk1.8.0:1.8.0 /usr/local/jenkins/java/jdk1.7.0_79:1.7.0_79
|
| findbugs | v3.0.0 |
| mvninstall | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-mvninstall-hbase-hadoop2-compat.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-compile-hbase-hadoop2-compat-jdk1.8.0.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-compile-hbase-hadoop2-compat-jdk1.8.0.txt
|
| compile | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-compile-hbase-hadoop2-compat-jdk1.7.0_79.txt
|
| javac | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-compile-hbase-hadoop2-compat-jdk1.7.0_79.txt
|
| findbugs | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-findbugs-hbase-hadoop2-compat.txt
|
| unit | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/artifact/patchprocess/patch-unit-hbase-hadoop2-compat.txt
|
|  Test Results | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/testReport/ |
| modules | C: hbase-hadoop2-compat U: hbase-hadoop2-compat |
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/1726/console |
| Powered by | Apache Yetus 0.2.1   http://yetus.apache.org |


This message was automatically generated.



> Reduce allocation of objects in metrics
> ---------------------------------------
>
>                 Key: HBASE-15742
>                 URL: https://issues.apache.org/jira/browse/HBASE-15742
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 1.3.0, 1.2.1, 1.0.3, 1.1.4, 0.98.19
>            Reporter: Phil Yang
>            Assignee: Phil Yang
>             Fix For: 2.0.0, 1.3.0, 1.4.0, 1.2.2, 0.98.20
>
>         Attachments: HBASE-15742-branch-1-v1.patch, HBASE-15742-branch-1.1-v1.patch,
HBASE-15742-branch-1.2-v1.patch, HBASE-15742-v1.patch, HBASE-15742-v2.patch, HBASE-15742-v3.patch,
HBASE-15742-v4.patch
>
>
> We use JMX and o.a.h.metrics2 to do some metrics on regions, tables, region servers and
cluster. We use MetricsInfo to show the information of metrics, and we use Interns to cache
MetricsInfo objects because it won't be changed.
> However, in Interns there are some static values to limit the max cached objects. We
can only cache 2010 metrics, but we have dozens of metrics for one region and we have some
RS-level metrics in each RS and all metrics for all regions will be saved in master. So each
server will have thousands of metrics, and we can not cache most of them. When we collect
metrics by JMX, we will create many objects which can be avoid. It increases the pressure
of GC and JMX has some caching logic so the objects can not be removed immediately which increases
the pressure more.
> Interns is in Hadoop project, and I think the implementation is not suitable for HBase.
Because we can not know how many MetricsInfo we have, it depends on the number of regions.
And we can not set it unlimited because we should remove the objects whose region is split,
moved, or dropped. I think we can use Guava's cache with expireAfterAccess which is very simple
and convenient.
> So we can add a new Interns class in HBase project first, and put it to upstream later.
> Moreover, in MutableHistogram#snapshot we create same Strings each time, we can create
them only in the first time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message