hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <>
Subject Re: Review Request 34447: HIVE-10761 : Create codahale-based metrics system for Hive
Date Sun, 31 May 2015 05:19:40 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated May 31, 2015, 5:19 a.m.)

Review request for hive, Chao Sun, Jimmy Xiang, and Xuefu Zhang.


One more minor change to the metrics file-reporter, where we are writing the file first then
mv it, so reader won't see a partial file.  Mv should be atomic if same file-system.  Updating
tests for the same.

Bugs: HIVE-10761

Repository: hive-git


See JIRA for the motivation.  Summary: There is an existing metric system that uses some custom
model and hooked up to JMX reporting, codahale-based metrics system will be desirable for
standard model and reporting.

This adds a codahale-based metrics system to HiveServer2 and HiveMetastore.  Metrics implementation
is now internally pluggable, and the existing Metrics system can be re-enabled by configuration
if desired for backward-compatibility.

Following metrics are supported by Metrics system:
1.  JVMPauseMonitor (used to call Hadoop's internal implementation, now forked off to integrate
with Metrics system)
2.  HMS API calls
3.  Standard JVM metrics (only for new implementation, as its free with codahale).

The following metrics reporting are supported by new system (configuration exposed)
1.  JMX
3.  JSON_FILE (periodic file of metrics that gets overwritten).

A goal is to add a webserver that exposes the JSON metrics, but this will defer to a later

Diffs (updated)

  common/pom.xml a615c1e 
  common/src/java/org/apache/hadoop/hive/common/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/ 01c9d1d 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/ PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/ PRE-CREATION

  common/src/java/org/apache/hadoop/hive/conf/ 49b8f97 
  common/src/test/org/apache/hadoop/hive/common/metrics/ PRE-CREATION

  common/src/test/org/apache/hadoop/hive/common/metrics/ e85d3f8 
  metastore/src/java/org/apache/hadoop/hive/metastore/ d81c856 
  pom.xml b21d894 
  service/src/java/org/apache/hive/service/server/ 58e8e49 
  shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/ 6d8166c 
  shims/0.23/src/main/java/org/apache/hadoop/hive/shims/ 19324b8 
  shims/common/src/main/java/org/apache/hadoop/hive/shims/ 5a6bc44 



New unit test added.  Manually tested.


Szehon Ho

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message