hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4632) Use hadoop counter as a stat publisher
Date Fri, 15 Nov 2013 22:51:21 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824206#comment-13824206
] 

Phabricator commented on HIVE-4632:
-----------------------------------

ashutoshc has requested changes to the revision "HIVE-4632 [jira] Use hadoop counter as a
stat publisher".

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregator.java:43 Instead of new
JobConf(hconf, ExecDriver.class), doing (JobConf) hconf is better, ie instead of creating
new JobConf object, cast hconf to JobConf.
  ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsPublisher.java:48 I don't think
we need to do this check isDecimals(). Counters by definition are countable, so they must
be of numeric type. If they are not, than its bug, which we shouldn't hide.
  Thing is this method will be called on every row, so its lots of wasted compute here.
  ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregator.java:69 Good to do LOG.error(e)
here.

REVISION DETAIL
  https://reviews.facebook.net/D11001

BRANCH
  HIVE-4632

ARCANIST PROJECT
  hive

To: JIRA, ashutoshc, navis
Cc: ashutoshc


> Use hadoop counter as a stat publisher
> --------------------------------------
>
>                 Key: HIVE-4632
>                 URL: https://issues.apache.org/jira/browse/HIVE-4632
>             Project: Hive
>          Issue Type: Improvement
>          Components: Statistics
>    Affects Versions: 0.12.0
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Minor
>         Attachments: D11001.2.patch, HIVE-4632.D11001.1.patch
>
>
> Currently stats are all long/aggregation type and can be safely acquired by hadoop counter
without other db or hbase.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message