hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-8456) Support Hive Counter to collect spark job metric[Spark Branch]
Date Thu, 16 Oct 2014 06:00:45 GMT

     [ https://issues.apache.org/jira/browse/HIVE-8456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chengxiang Li updated HIVE-8456:
--------------------------------
    Attachment: HIVE-8456.3-spark.patch

set javaSparkContext and hiveConf as transient variables, as they are only used in driver
side.

> Support Hive Counter to collect spark job metric[Spark Branch]
> --------------------------------------------------------------
>
>                 Key: HIVE-8456
>                 URL: https://issues.apache.org/jira/browse/HIVE-8456
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Chengxiang Li
>              Labels: Spark-M3
>         Attachments: HIVE-8456.1-spark.patch, HIVE-8456.2-spark.patch, HIVE-8456.3-spark.patch
>
>
> Several Hive query metric in Hive operators is collected by Hive Counter, such as CREATEDFILES
and DESERIALIZE_ERRORS, Besides, Hive use Counter as an option to collect table stats info.
 Spark support Accumulator which is pretty similiar with Hive Counter, we could try to enable
Hive Counter based on it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message