hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun" <>
Subject Review Request 24127: Research to use groupby transformation to replace Hive existing partitionByKey and SparkCollector combination
Date Thu, 31 Jul 2014 00:00:10 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hive.

Repository: hive-git


An attempt to fix the last patch by moving groupBy op to ShuffleTran.
Also, since now SparkTran::transform may have input/output value types other than BytesWritable,
we need to make it generic as well..
Also added a CompTran class, which is basically a composition of transformations. It offers
better type compatibility than ChainedTran.
This is NOT the perfect solution, and may subject to further change.


  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 4991568 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 01a70e9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 841db87 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 98d08e6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ d1af86d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 33e7d45 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ cf85af1 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 440dd93 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/ 6aa732f 




Chao Sun

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message