hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-15742) o.a.h.metrics2.lib.Interns cache only 2010 MetricsInfo
Date Sat, 30 Apr 2016 14:44:12 GMT
Phil Yang created HBASE-15742:

             Summary: o.a.h.metrics2.lib.Interns cache only 2010 MetricsInfo
                 Key: HBASE-15742
                 URL: https://issues.apache.org/jira/browse/HBASE-15742
             Project: HBase
          Issue Type: Bug
    Affects Versions: 0.98.19, 1.1.4, 1.0.3, 1.2.1, 1.3.0
            Reporter: Phil Yang

We use JMX and o.a.h.metrics2 to do some metrics on regions, tables, region servers and cluster.
We use MetricsInfo to show the information of metrics, and we use Interns to cache MetricsInfo
objects because it won't be changed.

However, in Interns there are some static values to limit the max cached objects. We can only
cache 2010 metrics, but we have dozens of metrics for one region and we have some RS-level
metrics in each RS and all metrics for all regions will be saved in master. So each server
will have thousands of metrics, and we can not cache most of them. When we collect metrics
by JMX, we will create many objects which can be avoid. It increases the pressure of GC and
JMX has some caching logic so the objects can not be removed immediately which increases the
pressure more.

Interns is in Hadoop project, and I think the implementation is not suitable for HBase. Because
we can not know how many MetricsInfo we have, it depends on the number of regions. And we
can not set it unlimited because we should remove the objects whose region is split, moved,
or dropped. I think we can use Guava's cache with expireAfterAccess which is very simple and

So we can add a new Interns class in HBase project first, and put it to upstream later.

This message was sent by Atlassian JIRA

View raw message