hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3102) Enhance HBase rMetrics for Long-running Stats
Date Fri, 22 Oct 2010 01:52:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12923735#action_12923735
] 

HBase Review Board commented on HBASE-3102:
-------------------------------------------

Message from: "Nicolas" <nspiegelberg@facebook.com>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1055/
-----------------------------------------------------------

(Updated 2010-10-21 18:50:11.325121)


Review request for hbase.


Changes
-------

  1. changed *Length -> *Size
  2. Added Size metrics for Compactions + Flushes

* Note: One Hackathony thing is that Hadoop's MetricsTimeVaryingRate appends 'mintime', 'maxtime',
etc to everything despite billing itself as a purpose-agnostic util.


Summary
-------

Example: a useful metric to observe track is compaction count + duration. Since compactions
are long running and only happen infrequently, the avg/opcount stats are should be reset longer
than the default polling period (5 sec). In addition to 'hbase.period', we should allow a
different duration after which long-running metrics should expire. This would also fix our
existing metrics problem where min/max stats are never reset until the process is restarted/upgraded.

This diff adds the capability for long-running metrics in addition to the infrastructure necessary
to log compactions + log splits.


This addresses bug HBASE-3102.
    http://issues.apache.org/jira/browse/HBASE-3102


Diffs (updated)
-----

  trunk/conf/hadoop-metrics.properties 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/master/HMaster.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/master/MasterFileSystem.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/master/metrics/MasterMetrics.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/metrics/PersistentMetricsTimeVaryingRate.java
PRE-CREATION 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/CompactSplitThread.java 1025790

  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/MemStoreFlusher.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/RegionServerMetrics.java
1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLog.java 1025790 
  trunk/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java 1025790 
  trunk/src/test/java/org/apache/hadoop/hbase/master/TestCatalogJanitor.java 1025790 

Diff: http://review.cloudera.org/r/1055/diff


Testing
-------

mvn clean install
cluster load test
** Note: this was primarily tested on 0.89.  Minimal testing on 0.90


Thanks,

Nicolas




> Enhance HBase rMetrics for Long-running Stats
> ---------------------------------------------
>
>                 Key: HBASE-3102
>                 URL: https://issues.apache.org/jira/browse/HBASE-3102
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>             Fix For: 0.90.0
>
>         Attachments: HBASE-3102_90.patch
>
>
> Example: a useful metric to observe track is compaction count + duration. Since compactions
are long running and only happen infrequently, the avg/opcount stats are should be reset longer
than the default polling period (5 sec). In addition to 'hbase.period', we should allow a
different duration after which long-running metrics should expire. This would also fix our
existing metrics problem where min/max stats are never reset until the process is restarted/upgraded.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message