pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kelly zhang <liyun.zh...@intel.com>
Subject Re: Review Request 43044: PIG-4766 Ensure GroupBy is optimized for all algebraic Operations
Date Thu, 04 Feb 2016 08:56:28 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/43044/#review117779
-----------------------------------------------------------




src/org/apache/pig/backend/hadoop/executionengine/spark/converter/ReduceByConverter.java (line
181)
<https://reviews.apache.org/r/43044/#comment179047>

    can we consider all the tuples with null key are same?
    
    I explain the detail in jira page.



src/org/apache/pig/data/SelfSpillBag.java (line 55)
<https://reviews.apache.org/r/43044/#comment179046>

    This modification is checked in PIG-4611.


- kelly zhang


On Feb. 3, 2016, 6:23 a.m., Pallavi Rao wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/43044/
> -----------------------------------------------------------
> 
> (Updated Feb. 3, 2016, 6:23 a.m.)
> 
> 
> Review request for pig, Xianda Ke, liyun zhang, Mohit Sabharwal, and Xuefu Zhang.
> 
> 
> Bugs: PIG-4766
>     https://issues.apache.org/jira/browse/PIG-4766
> 
> 
> Repository: pig-git
> 
> 
> Description
> -------
> 
> PIG-4709 introduced Combiner optimization for Group By. However, the patch did not handle
cases where constant/conditional expressions were used. It also did not handle limit.
> 
> This patch is to address those gaps.
> 
> 
> Diffs
> -----
> 
>   src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/expressionOperators/PORelationToExprProject.java
5fb49e2 
>   src/org/apache/pig/backend/hadoop/executionengine/spark/converter/ReduceByConverter.java
d4b521a 
>   src/org/apache/pig/backend/hadoop/executionengine/spark/optimizer/CombinerOptimizer.java
a05d009 
>   src/org/apache/pig/backend/hadoop/executionengine/util/CombinerOptimizerUtil.java 5c0919f

>   src/org/apache/pig/data/SelfSpillBag.java 4e08b99 
>   test/org/apache/pig/newplan/logical/relational/TestLocationInPhysicalPlan.java 0e45434

>   test/org/apache/pig/test/TestCombiner.java b2e81ac 
> 
> Diff: https://reviews.apache.org/r/43044/diff/
> 
> 
> Testing
> -------
> 
> With this patch, all tests in TestCombiner pass.
> 
> 
> Thanks,
> 
> Pallavi Rao
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message