hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chengxiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-7552) Collect spark job statistic through spark metrics [Spark Branch]
Date Fri, 05 Dec 2014 02:18:12 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chengxiang Li resolved HIVE-7552.
---------------------------------
    Resolution: Implemented

> Collect spark job statistic through spark metrics [Spark Branch]
> ----------------------------------------------------------------
>
>                 Key: HIVE-7552
>                 URL: https://issues.apache.org/jira/browse/HIVE-7552
>             Project: Hive
>          Issue Type: New Feature
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Na Yang
>
> MR/Tez use counters to collect job statistic information, while Spark does not use accumulator
to do the same thing. Instead, Spark store task metrics information in TaskMetrics and send
it back to scheduler. We  could get spark job statistic information through combine all TaskMetrics
with SparkListener.
> NO PRECOMMIT TESTS. This is for spark-branch only.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message