hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Remus Rusanu" <rem...@microsoft.com>
Subject Re: Review Request: HIVE-4612 Fix vector aggregates int type key output
Date Mon, 03 Jun 2013 14:08:52 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11427/
-----------------------------------------------------------

(Updated June 3, 2013, 2:08 p.m.)


Review request for hive.


Changes
-------

Added support for all current supported types (tinyint, smallint, int, bigint, boolean, timestamp,
string, float, double)


Description
-------

The VectorHashKeyValue output for int key type was broken, the M/R expects the type emitted
to match the type reduced. By using a BinaryWriter with a LongWritable instead of a IntWritable
the value was effectively corrupted.


This addresses bug HIVE-4612.
    https://issues.apache.org/jira/browse/HIVE-4612


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/VectorHashKeyWrapperBatch.java cd57151 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampUtils.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java 91366dd 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorReduceSinkOperator.java f61fcb6

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java 6bb5618 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedBatchUtil.java ffd7ef2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java aeff313 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriter.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriterFactory.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgDouble.java
54102a4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFAvgLong.java
8c6844b 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopDouble.java
a4084b0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdPopLong.java
28fdb36 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampDouble.java
4fa52ff 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFStdSampLong.java
551ae8a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumDouble.java
a2e8fb3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFSumLong.java
71b2e3d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopDouble.java
2dfbfa3 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarPopLong.java
de4811d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampDouble.java
5a21f44 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/gen/VectorUDAFVarSampLong.java
7b88c4f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFAvg.txt
d85346d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/VectorUDAFVar.txt
daae57b 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/FakeVectorRowBatchFromObjectIterables.java
6824ee7 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java 6fc230f


Diff: https://reviews.apache.org/r/11427/diff/


Testing
-------

manual test query


Thanks,

Remus Rusanu


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message