hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <>
Subject Re: Review Request 34447: HIVE-10761 : Create codahale-based metrics system for Hive
Date Wed, 27 May 2015 02:04:22 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated May 27, 2015, 2:04 a.m.)

Review request for hive, Chao Sun, Jimmy Xiang, and Xuefu Zhang.


Some changes: Make the metrics take in a list of configured reporters, add a new E2E test
for the basic Metastore metrics.

Bugs: HIVE-10761

Repository: hive-git


See JIRA for the motivation.  Summary: There is an existing metric system that uses some custom
model and hooked up to JMX reporting, codahale-based metrics system will be desirable for
standard model and reporting.

This adds a codahale-based metrics system to HiveServer2 and HiveMetastore.  Metrics implementation
is now internally pluggable, and the existing Metrics system can be re-enabled by configuration
if desired for backward-compatibility.

Following metrics are supported by Metrics system:
1.  JVMPauseMonitor (used to call Hadoop's internal implementation, now forked off to integrate
with Metrics system)
2.  HMS API calls
3.  Standard JVM metrics (only for new implementation, as its free with codahale).

The following metrics reporting are supported by new system (configuration exposed)
1.  JMX
3.  JSON_FILE (periodic file of metrics that gets overwritten).

A goal is to add a webserver that exposes the JSON metrics, but this will defer to a later

Diffs (updated)

  common/pom.xml a615c1e 
  common/src/java/org/apache/hadoop/hive/common/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/ 01c9d1d 
  common/src/java/org/apache/hadoop/hive/common/metrics/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/conf/ eff4d30 
  common/src/test/org/apache/hadoop/hive/common/metrics/ e85d3f8 
  common/src/test/org/apache/hadoop/hive/common/metrics/ PRE-CREATION

  common/src/test/org/apache/hadoop/hive/common/metrics/metrics2/ PRE-CREATION

  metastore/src/java/org/apache/hadoop/hive/metastore/ e796048 
  pom.xml 920f7a5 
  service/src/java/org/apache/hive/service/server/ 58e8e49 
  shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/ 6d8166c 
  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/ d349068 
  shims/common/src/main/java/org/apache/hadoop/hive/shims/ 5a6bc44 



New unit test added.  Manually tested.


Szehon Ho

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message