pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park" <piaozhe...@gmail.com>
Subject Re: Review Request 16165: PIG-3618 Replace broadcast edges with scatter/gather edges in union
Date Wed, 11 Dec 2013 22:47:07 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Dec. 11, 2013, 10:47 p.m.)

Review request for pig, Alex Bain, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.


Got rid of queue in POUnionTezLoad for efficiency. Added a TODO comment for TEZ-661.

Bugs: PIG-3618

Repository: pig-git


Replace broadcast edges with scatter/gather edges for union. I also changed the name of POPackage
from POBroadcastTezLoad to POUnionTezLoad.

Diffs (updated)

  src/org/apache/pig/backend/hadoop/executionengine/tez/POBroadcastTezLoad.java d7b9d5a 
  src/org/apache/pig/backend/hadoop/executionengine/tez/POLocalRearrangeTez.java 2a96da8 
  src/org/apache/pig/backend/hadoop/executionengine/tez/POUnionTezLoad.java e69de29 
  src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java 8fb18d3 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezCompiler.java ea4d167 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java 2e55ea9 
  src/org/apache/pig/backend/hadoop/executionengine/tez/TezPOPackageAnnotator.java 79c97f7

  test/e2e/pig/tests/tez.conf b7dd862 

Diff: https://reviews.apache.org/r/16165/diff/


Ran tez unit tests and e2e tests.

I set the parallel of union in the e2e test case to 2.


Cheolsoo Park

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message