pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cheolsoo Park" <piaozhe...@gmail.com>
Subject Re: Review Request 17439: PIG-3728: Fix TestSkewedJoin unit test in tez mode
Date Tue, 28 Jan 2014 03:44:39 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 28, 2014, 3:44 a.m.)

Review request for pig, Alex Bain, Daniel Dai, Mark Wagner, and Rohini Palaniswamy.


Minor clean up.

Bugs: PIG-3728

Repository: pig-git


The patch fixes TestSkewedJoin in tez mode. Note I changed SkewedPartitionerTez and WeightedRangePartitionerTez
to allow null sampleMap to make testSkewedJoinEmptyInput pass. It used to throw a runtime
exception, but now it assumes empty sample input.

I also fixed MiniCluster in hadoop20. They were missing the getExecType() method.

Diffs (updated)

  shims/test/hadoop20/org/apache/pig/test/MiniCluster.java 3e64a5d 
  shims/test/hadoop20/org/apache/pig/test/TezMiniCluster.java 98a580c 
  src/org/apache/pig/backend/hadoop/executionengine/tez/POPartitionRearrangeTez.java b17b38b

  src/org/apache/pig/backend/hadoop/executionengine/tez/PigProcessor.java 2b6c899 
  src/org/apache/pig/backend/hadoop/executionengine/tez/SkewedPartitionerTez.java 5fe202b

  src/org/apache/pig/backend/hadoop/executionengine/tez/WeightedRangePartitionerTez.java 36441fb

  test/org/apache/pig/test/TestAccumulator.java 516d5c5 
  test/org/apache/pig/test/TestSkewedJoin.java be91d6b 
  test/tez-tests 0d32d23 

Diff: https://reviews.apache.org/r/17439/diff/


* TestSkewedJoin passes in both tez and mr mode.
* ant test-tez passes except TestCustomPartitioner (known).
* All e2e tests pass.


Cheolsoo Park

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message