hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-7552) Collect spark job statistic through spark metrics [Spark Branch]
Date Tue, 19 Aug 2014 03:27:18 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Brock Noland updated HIVE-7552:
-------------------------------

    Summary: Collect spark job statistic through spark metrics [Spark Branch]  (was: Collect
spark job statistic through spark metrics[Spark Branch])

> Collect spark job statistic through spark metrics [Spark Branch]
> ----------------------------------------------------------------
>
>                 Key: HIVE-7552
>                 URL: https://issues.apache.org/jira/browse/HIVE-7552
>             Project: Hive
>          Issue Type: New Feature
>          Components: Spark
>            Reporter: Chengxiang Li
>            Assignee: Na Yang
>
> MR/Tez use counters to collect job statistic information, while Spark does not use accumulator
to do the same thing. Instead, Spark store task metrics information in TaskMetrics and send
it back to scheduler. We  could get spark job statistic information through combine all TaskMetrics
with SparkListener.
> NO PRECOMMIT TESTS. This is for spark-branch only.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message