hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10436) hbase 0.96+ jmx metrics do not have regionserver info any more.
Date Wed, 29 Jan 2014 12:56:09 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13885308#comment-13885308
] 

Jonathan Hsieh commented on HBASE-10436:
----------------------------------------

bq. First this patch only addresses jmx and not the actual metrics system. (See how the other
HMaster metrics are exposed; Interfaces in hbase-hadoop-compat and implementations in hbase-hadoop1|2-compat)

This is intentional and actually the point.  It is pretty clear from the JMX (java management
extensions) documentation [1] that it is a *management* interface - not just a *metrics* interface.
 It is intended to be a common interface for managing java services.  Services could be designed
to accept input and provide information all via the JMX interface and could be designed (if
facilities were provided) to manage and runtime configuration changes.

I do agree that I should remove the word "metrics" from the jira name.  

bq. Secondly, This will synchronize on the online servers list everytime that JMX is polled.
That seems pretty onerous for exposing the list of region servers. That api already exists;
We should have one way of getting that list, not multiple.

Having separate jmx modules allows the user to selectively poll information and thus avoid
any cost associated with gathering the online regionserver list [2].  Furthermore, this is
at the master and I contend that any synchronization would actually be fairly cheap -- I don't
believe there is evidence that there is high contention on this online regionserver list.
 This only retrieves info when polled - its is not a "NotificationEmmitter".   If it was truly
onerous, I would make the same argument that we shouldn't provide this info in the hbase master
webpage and we dare not auto refresh the page (what a NotificationEmitter would do) because
of the synchronization costs.  

bq. Thirdly I disagree that the metrics api should be treated like the user api.

JMX is a management api protocol not just meant to server metrics.

Do external programs plug into this api?  Does a user have to change code if the api changes?
 yes. 
Should users need to significantly modify their applications when deciding to upgrade?  ideally
no.
There is a concept of versioning apis and metrics -- ideally we should have used that approach.
 Instead for this transition our users would have to validate "modelerType" and "name" attributes.


bq. We told users the metrics would change very early in 0.95

Removal is different is generally called out and not just a change.  We preserved functionality
when we changed the signature of methods to use ServerName. We didn't change rowlocks, we
removed them. 

Regarding the examples -- each hlog or memstore implementation would have a different bean.
 The refactoring that split the metrics into separate beans was a really good thing -- I believe
that IPC, memstore and hlog is already split out into separate jmx beans.  This split allows
for different flavors with different implementations.  I would hope that there would be common
attributes regardless of what kind of future hlogs come to be -- (# of kvs or waledits written
will always be hlog concepts regardless of its implementation details).  

[1]  http://docs.oracle.com/javase/tutorial/jmx/overview/index.html
[2] http://docs.oracle.com/cd/E13222_01/wls/docs100/jmxinst/understanding.html#wp1100632

> hbase 0.96+ jmx metrics do not have regionserver info any more.
> ---------------------------------------------------------------
>
>                 Key: HBASE-10436
>                 URL: https://issues.apache.org/jira/browse/HBASE-10436
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.98.0, 0.96.0, 0.99.0
>            Reporter: Jonathan Hsieh
>            Assignee: Jonathan Hsieh
>            Priority: Critical
>         Attachments: hbase-10436.patch
>
>
> HBase 0.96's refactored jmx beans do not contain the master's list of dead region servers
and live regionservers with load info.  HBase 0.94 did (though in a single monolithic blob).
 
> This JMX interface should be considered as much of an API as the the normal wire or java
api.  Dropping values from this was done without deprecation and the removal of this information
is a functional regression.
> We should provide the information in the 0.96+ JMX.  HBase 0.94 had a  monolithic JMX
blob ("hadoop:service=Master,name=Master")  that contained a lot of information, including
the regionserver list and the cached regionserver load for each region  found on the master
webpage.  0.96+ refactored jmx this into several jmx beans which could be selectively retrieved.
 These include:
> * hadoop:service=HBase,name=Master,sub=AssignmentManager
> * hadoop:service=HBase,name=Master,sub=Balancer
> * hadoop:service=HBase,name=Master,sub=Server
> * hadoop:service=HBase,name=Master,sub=FileSystem
> Specifically the (Hadoop:service=HBase,name=Master,sub=Server) listing that used to contain
regionservers and deadregionservers in jmx were replaced in   with numRegionServers and numDeadRegionservers
which only contain counts.  
> I propose just adding another mbean called "RegionServers" under the bean: "hadoop:service=HBase,name=Master,sub=RegionServers"



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message