hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alex Baranau (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4050) Update HBase metrics framework to metrics2 framework
Date Wed, 18 Jul 2012 22:52:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13417793#comment-13417793

Alex Baranau commented on HBASE-4050:

Looking at adding MetricSources for Master and RegionServer and have some Qs.

1) MetricsSystem - always has name "hbase"? What if we want to try single-node cluster? Would
be great to separate metrics systems. E.g. in Hadoop code I see "datanode", "namenode", …

2) I was thinking to extract maps (counters & gauges) into separate class along with the
code that manages them. I.e. in new hbase.MetricsRegistry. We can add then Tags in there,
etc. I.e. this would be pretty much similar to org.apache.hadoop.metrics2.lib.MetricsRegistry,
but will allow also removing metrics (also read the last piece of comment to see why we might
want it as well). Thoughts? 

3) Does it make sense to create interfaces for MetricMutableGaugeLong and MetricMutableCounterLong
and allow MetricSource implementations to create & obtain them? So that they could be
fields in MetricSource classes. This would allow at least:
* access metrics much faster, without lookup in Map
* this would remove "proxy methods" from BaseMetricsSourceImpl (or from MetricsRegistry, after
extracting it) and will make easier to provide other implementations (in addition to counter
& gauge), independently (i.e. without adding proxy methods in BaseMetricsSourceImpl).

4) In case of RegionServer and Master metrics I need to pass initialization parameters. By
analogy with DataNode metrics, I'd pass servername, sessionId. As we use ServiceLoader, there's
no way to pass parameters to constructor. I was thinking about adding smth like init() method
to the MasterMetricsSource (which is in common compat module), but looks not very nice and
will not actually work very well with current BaseMetricsSource implementation as things are
initialized in constructor there. I.e. I will have to change that and move things into init()
method (doesn't look nice to me).

Another thought. What if slightly "move" what is placed in compat module? What if we put there
more "lower-level" intruments. We could: 
* extract hbase.MetricRegistry as per above, make common interface for it and place in hadoop
compat module, provide factory for creating hbase.MetricsRegistry there as well
* define hbase.MetricSource interface with single method getMetricRegistry()
* create service wrapper of DefaultMetricsSystem, which allows to register hbase.MetricsSource
  (in each hadoop1/2-compat modules hbase.MetricsSource.getMetricRegistry() can be used to
implement hadoop.MetricSource's getMetrics() method via getting data from hbase.MetricsRegistry)

This way we will offer to MetricsSource implementations (in main hbase modules) lower-level
tools and will be more flexible (e.g. with passing parameters to MetricSource implementation
ctors). Also, MetricSource implementations (in main modules) will look closer to what was
intended by Hadoop Metrics framework (judging by datanode/namenode metric sources). We could
do what is mentioned in 3rd point above. E.g. typical metricSource will look like: 

public class HMasterMetrics implements MetricsSource {
  final String name;
  final MetricsRegistry registry = MetricsRegistryFactory.create("hmaster");

  final MutableCounterLong clusterRequests =
          registry.newCounter("cluster_requests", "", 0);

  public HMasterMetrics(final String name, final String sessionId) {
    this.name = name;
    JvmMetricsSourceFactory.create("HMaster", sessionId);
    registry.setContext(name).tag("sessionId", "", sessionId);

  public MetricsRegistry getMetrics() {
    return registry;

  public void incClusterRequests(int delta) {

Also, it will be easier to move out wrom using this shim when it is no longer needed by easy
switching these shim hbase.MetricsRegistry, hbase.MetricsSource, hbase.DefaultMetricsSystem
classes to hadoop.* ones.

I see a lot of benefits, but may be I'm missing smth. The only fear is that by going with
lower-level tools shims we might end up having more of these shim classes/interfaces (like
adding shim MetricMutableGaugeLong and MetricMutableCounterLong interfaces in this case so
far). Though this might not be that bad.

Does it makes sense to you guys at all?

I hope I'm not acting counter-productive by suggesting changes to things that already work
(the patch by Elliott)
> Update HBase metrics framework to metrics2 framework
> ----------------------------------------------------
>                 Key: HBASE-4050
>                 URL: https://issues.apache.org/jira/browse/HBASE-4050
>             Project: HBase
>          Issue Type: New Feature
>          Components: metrics
>    Affects Versions: 0.90.4
>         Environment: Java 6
>            Reporter: Eric Yang
>            Assignee: Elliott Clark
>            Priority: Critical
>             Fix For: 0.96.0
>         Attachments: 4050-metrics-v2.patch, 4050-metrics-v3.patch, HBASE-4050-0.patch,
HBASE-4050-1.patch, HBASE-4050-2.patch, HBASE-4050-3.patch, HBASE-4050-5.patch, HBASE-4050-6.patch,
HBASE-4050-7.patch, HBASE-4050-8.patch, HBASE-4050.patch
> Metrics Framework has been marked deprecated in Hadoop 0.20.203+ and 0.22+, and it might
get removed in future Hadoop release.  Hence, HBase needs to revise the dependency of MetricsContext
to use Metrics2 framework.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message