hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Hanson" <eh...@microsoft.com>
Subject Re: Review Request: Column Column, and Column Scalar vectorized execution tests
Date Fri, 17 May 2013 23:32:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11133/#review20739
-----------------------------------------------------------

Ship it!


Ship It!

- Eric Hanson


On May 17, 2013, 6:06 p.m., tony murphy wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/11133/
> -----------------------------------------------------------
> 
> (Updated May 17, 2013, 6:06 p.m.)
> 
> 
> Review request for hive, Jitendra Pandey, Eric Hanson, Sarvesh Sakalanaga, and Remus
Rusanu.
> 
> 
> Description
> -------
> 
> This patch adds Column Column, and Column Scalar vectorized execution tests. These tests
are generated in parallel with the vectorized expressions. The tests focus is on validating
the column vector and the vectorized row batch metadata regarding nulls, repeating, and selection.
> 
> Overview of Changes:
> 
> CodeGen.java:
> + joinPath, getCamelCaseType, readFile and writeFile made static for use in TestCodeGen.java.
> + filter types now specify null as their output type rather than "doesn't matter" to
make detection for test generation easier.
> + support for test generation added.
> 
> TestCodeGen.java & Templates: 
>      TestClass.txt
>      TestColumnColumnFilterVectorExpressionEvaluation.txt,
>      TestColumnColumnOperationVectorExpressionEvaluation.txt,
>      TestColumnScalarFilterVectorExpressionEvaluation.txt,
>      TestColumnScalarOperationVectorExpressionEvaluation.txt
> +This class is mutable and maintains a hashmap of TestSuiteClassName to test cases. The
tests cases are added over the course of vectorized expressions class generation, with test
classes being outputted at the end. For each column vector (inputs and/or outputs) a matrix
of pairwise covering Booleans is used to generate test cases across nulls and repeating dimensions.
Based on the input column vector(s) nulls and repeating states the states of the output column
vector (if there is one) is validated, along with the null vector. For filter operations the
selection vector is validated against the generated data. Each template corresponds to a class
representing a test suite.
> 
> VectorizedRowGroupUtil.java
> +added methods generateLongColumnVector and generateDoubleColumnVector for generating
the respective column vectors with optional nulls and/or repeating values.
> 
> 
> This addresses bug HIVE-4553.
>     https://issues.apache.org/jira/browse/HIVE-4553
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/CodeGen.java
53d9a7a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestClass.txt
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestCodeGen.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnFilterVectorExpressionEvaluation.txt
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnColumnOperationVectorExpressionEvaluation.txt
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarFilterVectorExpressionEvaluation.txt
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/templates/TestColumnScalarOperationVectorExpressionEvaluation.txt
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnColumnFilterVectorExpressionEvaluation.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnColumnOperationVectorExpressionEvaluation.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnScalarFilterVectorExpressionEvaluation.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/gen/TestColumnScalarOperationVectorExpressionEvaluation.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/VectorizedRowGroupGenUtil.java
8a07567 
> 
> Diff: https://reviews.apache.org/r/11133/diff/
> 
> 
> Testing
> -------
> 
> generated tests, and ran them.
> 
> 
> Thanks,
> 
> tony murphy
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message