hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang" <xzh...@cloudera.com>
Subject Re: Review Request 27720: HIVE-8777 should only register used operator counters.[Spark Branch]
Date Fri, 07 Nov 2014 14:36:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27720/#review60316
-----------------------------------------------------------


1. Looking at the diff, I was sure where we are removing unnecessary counter registrations.
2. It would be great if we can have some tests that are enabled with counter statistics collection,
so that we know what kind of output we are expecting and avoid future breakage.

- Xuefu Zhang


On Nov. 7, 2014, 5:27 a.m., chengxiang li wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27720/
> -----------------------------------------------------------
> 
> (Updated Nov. 7, 2014, 5:27 a.m.)
> 
> 
> Review request for hive and Xuefu Zhang.
> 
> 
> Bugs: HIVE-8777
>     https://issues.apache.org/jira/browse/HIVE-8777
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Currently we register all hive operator counters in SparkCounters, while actually not
all hive operators are used in SparkTask, we should iterate SparkTask's operators, and only
register conuters required.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkClient.java e955da3 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 46b04bc 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/spark/counter/SparkCounters.java bb3597a

>   ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java 66fd6b6 
> 
> Diff: https://reviews.apache.org/r/27720/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> chengxiang li
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message