hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Surówka (JIRA) <j...@apache.org>
Subject [jira] [Created] (HIVE-2446) Introduction of client statistics publishers possibility
Date Wed, 14 Sep 2011 21:24:09 GMT
Introduction of client statistics publishers possibility
--------------------------------------------------------

                 Key: HIVE-2446
                 URL: https://issues.apache.org/jira/browse/HIVE-2446
             Project: Hive
          Issue Type: Improvement
          Components: Clients, Statistics
            Reporter: Robert Surówka
            Priority: Minor


The purpose of this change is to allow publication or storage of counters while the job is
running.

Introduced two new variables to hive-default.xml and HiveConf.java: "hive.client.stats.publishers"
and "hive.client.stats.counters". First one specifies classes names, whose instances will
be executed by HadoopJobExecHelper.java (similarly as hooks are) in its method progress(ExecDriverTaskHandle):
MapRedStats. Second one specifies list of counters that any client stat publishers should
publish or stored. Details regarding format of this list is up to a specific deployment (it
is up to client stats publishers to parse it), yet it is required to use display names of
counter groups and counters.

Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats package, that must
be implemented by all stats publishers.

Added code to progress(ExecDriverTaskHandle): MapRedStats from HadoopJobExecHelper.java that
puts counters' values to a Java map and then executes registered client stats publishers giving
them that map and running job id. Added two new methods to HadoopJobExecHelper: extractAllCounterValues(Counters)
and getClientStatsPublishers() that are used by code from previous sentence.

Made cosmetic changes in two other classes



 


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message