pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park" <piaozhe...@gmail.com>
Subject Re: Review Request 16165: PIG-3618 Replace broadcast edges with scatter/gather edges in union
Date Wed, 11 Dec 2013 22:47:07 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16165/
-----------------------------------------------------------

(Updated Dec. 11, 2013, 10:47 p.m.)


Review request for pig, Alex Bain, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.


Changes
-------

Got rid of queue in POUnionTezLoad for efficiency. Added a TODO comment for TEZ-661.


Bugs: PIG-3618
    https://issues.apache.org/jira/browse/PIG-3618


Repository: pig-git


Description
-------

Replace broadcast edges with scatter/gather edges for union. I also changed the name of POPackage
from POBroadcastTezLoad to POUnionTezLoad.


Diffs (updated)
-----

  src/org/apache/pig/backend/hadoop/executionengine/tez/POBroadcastTezLoad.java d7b9d5a 
  src/org/apache/pig/backend/hadoop/executionengine/tez/POLocalRearrangeTez.java 2a96da8 
  src/org/apache/pig/backend/hadoop/executionengine/tez/POUnionTezLoad.java e69de29 
  src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java 8fb18d3 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java ea4d167 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java 2e55ea9 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezPOPackageAnnotator.java 79c97f7

  test/e2e/pig/tests/tez.conf b7dd862 

Diff: https://reviews.apache.org/r/16165/diff/


Testing
-------

Ran tez unit tests and e2e tests.

I set the parallel of union in the e2e test case to 2.


Thanks,

Cheolsoo Park


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message