hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Re: Review Request 34447: HIVE-10761 : Create codahale-based metrics system for Hive
Date Tue, 19 May 2015 23:58:23 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34447/
-----------------------------------------------------------

(Updated May 19, 2015, 11:58 p.m.)


Review request for hive.


Bugs: HIVE-10761
    https://issues.apache.org/jira/browse/HIVE-10761


Repository: hive-git


Description
-------

See JIRA for the motivation.  Summary: There is an existing metric system that uses some custom
model and hooked up to JMX reporting, codahale-based metrics system will be desirable for
standard model and reporting.

This adds a codahale-based metrics system to HiveServer2 and HiveMetastore.  Metrics implementation
is now internally pluggable, and the existing Metrics system can be re-enabled by configuration
if desired for backward-compatibility.

Following metrics are supported by Metrics system:
1.  JVMPauseMonitor (used to call Hadoop's internal implementation, now forked off to integrate
with Metrics system)
2.  HMS API calls
3.  Standard JVM metrics (only for new implementation, as its free with codahale).

The following metrics reporting are supported by new system (configuration exposed)
1.  JMX
2.  CONSOLE
3.  JSON_FILE (periodic file of metrics that gets overwritten).

A goal is to add a webserver that exposes the JSON metrics, but this will defer to a later
implementation.


Diffs
-----

  common/pom.xml a615c1e 
  common/src/java/org/apache/hadoop/hive/common/JvmPauseMonitor.java PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/Metrics.java 01c9d1d 
  common/src/java/org/apache/hadoop/hive/common/metrics/MetricsLegacy.java PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/Metrics.java PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/MetricsFactory.java PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/Metrics.java PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/MetricsReporting.java PRE-CREATION

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java eff4d30 
  common/src/test/org/apache/hadoop/hive/common/metrics/TestMetrics.java e85d3f8 
  common/src/test/org/apache/hadoop/hive/common/metrics/TestMetricsLegacy.java PRE-CREATION

  common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/TestMetrics.java PRE-CREATION

  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java e796048 
  pom.xml 920f7a5 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 58e8e49 
  shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 6d8166c 
  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java d349068 
  shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java 5a6bc44 

Diff: https://reviews.apache.org/r/34447/diff/


Testing
-------

New unit test added.  Manually tested.


Thanks,

Szehon Ho


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message