hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Surówka (JIRA) <>
Subject [jira] [Created] (HIVE-2446) Introduction of client statistics publishers possibility
Date Wed, 14 Sep 2011 21:24:09 GMT
Introduction of client statistics publishers possibility

                 Key: HIVE-2446
             Project: Hive
          Issue Type: Improvement
          Components: Clients, Statistics
            Reporter: Robert Surówka
            Priority: Minor

The purpose of this change is to allow publication or storage of counters while the job is

Introduced two new variables to hive-default.xml and "hive.client.stats.publishers"
and "hive.client.stats.counters". First one specifies classes names, whose instances will
be executed by (similarly as hooks are) in its method progress(ExecDriverTaskHandle):
MapRedStats. Second one specifies list of counters that any client stat publishers should
publish or stored. Details regarding format of this list is up to a specific deployment (it
is up to client stats publishers to parse it), yet it is required to use display names of
counter groups and counters.

Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats package, that must
be implemented by all stats publishers.

Added code to progress(ExecDriverTaskHandle): MapRedStats from that
puts counters' values to a Java map and then executes registered client stats publishers giving
them that map and running job id. Added two new methods to HadoopJobExecHelper: extractAllCounterValues(Counters)
and getClientStatsPublishers() that are used by code from previous sentence.

Made cosmetic changes in two other classes


This message is automatically generated by JIRA.
For more information on JIRA, see:


View raw message