hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8456) Support Hive Counter to collect spark job metric[Spark Branch]
Date Wed, 15 Oct 2014 08:57:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14172158#comment-14172158
] 

Chengxiang Li commented on HIVE-8456:
-------------------------------------

As Spark does not use Counter to collect framework/job metric like MR/Tez, so Hive on Spark
only use Counter in several simple case, I implement a simple implementation which would fit
into those requirement based on Spark accumulator.

> Support Hive Counter to collect spark job metric[Spark Branch]
> --------------------------------------------------------------
>
>                 Key: HIVE-8456
>                 URL: https://issues.apache.org/jira/browse/HIVE-8456
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Chengxiang Li
>              Labels: Spark-M3
>         Attachments: HIVE-8456.1-spark.patch, HIVE-8456.2-spark.patch
>
>
> Several Hive query metric in Hive operators is collected by Hive Counter, such as CREATEDFILES
and DESERIALIZE_ERRORS, Besides, Hive use Counter as an option to collect table stats info.
 Spark support Accumulator which is pretty similiar with Hive Counter, we could try to enable
Hive Counter based on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message