hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Barna Zsombor Klara <>
Subject Re: Review Request 51193: HIVE-14358: Add metrics for number of queries executed for each execution engine (mr, spark, tez)
Date Thu, 18 Aug 2016 18:35:26 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Aug. 18, 2016, 6:35 p.m.)

Review request for hive.


As per Peter's comments, I have changed the import statements. Static imports have been moved
to the beginning, imported static methods/classes are enumerated instead of using the wildcard.

Repository: hive-git


HIVE-14358: Add metrics for number of queries executed for each execution engine (mr, spark,

Diffs (updated)

  common/src/java/org/apache/hadoop/hive/common/metrics/common/ 9dc96f9c6412720a891b5c55e2074049c893d780

  ql/src/java/org/apache/hadoop/hive/ql/ 183ed829ef1742e48539f8928293d56b77bc43c8

  ql/src/java/org/apache/hadoop/hive/ql/exec/ eeaa54320ffaa7ba5d6ebece80a0cb4aadc1dada

  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ ce1106d91db9ef75e7b425d5950f888bacbfb3e5

  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ ac922ce486babe042984d87a7f7442cbfc11484f

  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 0b494aa5548f8e6ae76e2d0eea9a7afb33961f97

  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/ 25c4514b34fb2ed4fc8b1238059bd9dc29d2741b

  ql/src/test/org/apache/hadoop/hive/ql/exec/mr/ PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/mr/ PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/spark/ PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/ 53672a9783b4d13c5eed4ef01f5c16af568a0a41



Ran the new unit tests in the ql project, everything was green.
Checked that the metrics for map reduce and spark tasks were appearing and being incremented
correctly using JMX. 
Map reduce tasks were being created by a simple select statement containing a join.
Spark tasks were being created by the same query with the spark execution engine being used.
The metrics were correct across several beeline connections, and were reset once the HiveServer2
was restarted.
The metric collection can be turned on/off using the configuration variable "hive.server2.metrics.enabled".
No errors/exceptions encountered when the metrics were disabled.

NB only the root tasks are incrementing the counter since the original jira was about counting
the number of queries issued against each exeution engine, so a complex query resulting in
more than one task should only count as one as per my understanding.


Barna Zsombor Klara

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message