hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Pandey" <jnpan...@gmail.com>
Subject Re: Review Request 14130: Merge vectorization branch to trunk
Date Wed, 25 Sep 2013 00:08:37 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14130/
-----------------------------------------------------------

(Updated Sept. 25, 2013, 12:08 a.m.)


Review request for hive and Ashutosh Chauhan.


Changes
-------

Updated patch, addressing the review comments and rebased to the latest state of the trunk.


Bugs: HIVE-5283
    https://issues.apache.org/jira/browse/HIVE-5283


Repository: hive-git


Description
-------

Merge vectorization branch to trunk.


Diffs
-----

  ant/src/org/apache/hadoop/hive/ant/GenVectorCode.java PRE-CREATION 
  ant/src/org/apache/hadoop/hive/ant/GenVectorTestCode.java PRE-CREATION 
  build-common.xml 5a16bff 
  build.xml cf75b3d 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e971644 
  conf/hive-default.xml.template 1ee756c 
  data/files/alltypesorc PRE-CREATION 
  ql/build.xml ac2b7b0 
  ql/src/gen/vectorization/ExpressionTemplates/ColumnArithmeticColumn.txt PRE-CREATION 
  ql/src/gen/vectorization/ExpressionTemplates/ColumnArithmeticScalar.txt PRE-CREATION 
  ql/src/gen/vectorization/ExpressionTemplates/ColumnCompareScalar.txt PRE-CREATION 
  ql/src/gen/vectorization/ExpressionTemplates/ColumnUnaryMinus.txt PRE-CREATION 
  ql/src/gen/vectorization/ExpressionTemplates/FilterColumnCompareColumn.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/FilterColumnCompareScalar.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/FilterScalarCompareColumn.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/FilterStringColumnCompareColumn.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/FilterStringColumnCompareScalar.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/FilterStringScalarCompareColumn.txt PRE-CREATION

  ql/src/gen/vectorization/ExpressionTemplates/ScalarArithmeticColumn.txt PRE-CREATION 
  ql/src/gen/vectorization/TestTemplates/TestClass.txt PRE-CREATION 
  ql/src/gen/vectorization/TestTemplates/TestColumnColumnFilterVectorExpressionEvaluation.txt
PRE-CREATION 
  ql/src/gen/vectorization/TestTemplates/TestColumnColumnOperationVectorExpressionEvaluation.txt
PRE-CREATION 
  ql/src/gen/vectorization/TestTemplates/TestColumnScalarFilterVectorExpressionEvaluation.txt
PRE-CREATION 
  ql/src/gen/vectorization/TestTemplates/TestColumnScalarOperationVectorExpressionEvaluation.txt
PRE-CREATION 
  ql/src/gen/vectorization/UDAFTemplates/VectorUDAFAvg.txt PRE-CREATION 
  ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMax.txt PRE-CREATION 
  ql/src/gen/vectorization/UDAFTemplates/VectorUDAFMinMaxString.txt PRE-CREATION 
  ql/src/gen/vectorization/UDAFTemplates/VectorUDAFSum.txt PRE-CREATION 
  ql/src/gen/vectorization/UDAFTemplates/VectorUDAFVar.txt PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java b7d45c5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java bcee201 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FilterOperator.java d2c981d 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java e498327 
  ql/src/java/org/apache/hadoop/hive/ql/exec/KeyWrapper.java c303b30 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java 3b15667 
  ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java 85a22b7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 1f763a4 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java c6cecb5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java bd9d0bb 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/ColumnVector.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/DoubleColumnVector.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/LongColumnVector.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/TimestampUtils.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorAggregationBufferBatch.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorAggregationBufferRow.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFileSinkOperator.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorFilterOperator.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapper.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorHashKeyWrapperBatch.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorReduceSinkOperator.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorSelectOperator.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizationContext.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedBatchUtil.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedColumnarSerDe.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedInputFormatInterface.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatch.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedRowBatchCtx.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorizedSerde.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColAndCol.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ColOrCol.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/ConstantVectorExpression.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterConstantBooleanVectorExpression.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprAndExpr.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterExprOrExpr.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterNotExpr.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/FilterStringColLikeStringScalar.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IdentityExpression.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNotNull.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/IsNull.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/LongColDivideLongColumn.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/LongColDivideLongScalar.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/LongScalarDivideLongColumn.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NotCol.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/NullUtil.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsFalse.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNotNull.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsNull.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/SelectColumnIsTrue.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringConcatColCol.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringConcatColScalar.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringConcatScalarCol.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringExpr.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLTrim.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLength.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringLower.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringRTrim.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringSubstrColStart.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringSubstrColStartLen.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringTrim.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDF.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUnaryUDFDirect.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/StringUpper.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpression.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriter.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorExpressionWriterFactory.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFDayOfMonthLong.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFHourLong.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFMinuteLong.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFMonthLong.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFSecondLong.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFTimestampFieldLong.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFUnixTimeStampLong.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFWeekOfYearLong.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/VectorUDFYearLong.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorAggregateExpression.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCount.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/expressions/aggregates/VectorUDAFCountStar.java
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/udf/VectorUDFAdaptor.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/exec/vector/udf/VectorUDFArgDesc.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/io/CommonRCFileInputFormat.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileInputFormat.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/io/VectorizedRCFileRecordReader.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/io/orc/BitFieldReader.java f0be6df 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/DynamicByteArray.java 248cea9 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/IntegerReader.java 04cfa58 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java 51863fc 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSerde.java 8b46370 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReader.java 05240ce 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 81c914c 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthByteReader.java 2fdd498 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReader.java b737b0f 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RunLengthIntegerReaderV2.java 4a4165f 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcSerde.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/PhysicalOptimizer.java 3038a98

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/Vectorizer.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 767f545 
  ql/src/java/org/apache/hadoop/hive/ql/plan/AbstractOperatorDesc.java 4bb28b5 
  ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeGenericFuncDesc.java 52fe47b 
  ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java 75d2ec9 
  ql/src/java/org/apache/hadoop/hive/ql/util/JavaDataModel.java e3eec02 
  ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 75b8a5f 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorFilterOperator.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorGroupByOperator.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorSelectOperator.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizationContext.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatch.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/TestVectorizedRowBatchCtx.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestConstantVectorExpression.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestUnaryMinus.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorArithmeticExpressions.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorExpressionWriters.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorFilterExpressions.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorLogicalExpressions.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorScalarColArithmetic.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorStringExpressions.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/expressions/TestVectorTimestampExpressions.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/udf/TestVectorUDFAdaptor.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/udf/generic/GenericUDFIsNull.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/udf/legacy/ConcatTextLongDoubleUDF.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/udf/legacy/LongUDF.java PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/AllVectorTypesRecord.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputDesc.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeCaptureOutputOperator.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperator.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorDataSourceOperatorDesc.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchBase.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromConcat.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromLongIterables.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromObjectIterables.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/FakeVectorRowBatchFromRepeats.java
PRE-CREATION 
  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/OrcFileGenerator.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/exec/vector/util/VectorizedRowGroupGenUtil.java PRE-CREATION

  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestVectorizedORCReader.java PRE-CREATION 
  ql/src/test/queries/clientpositive/vectorization_short_regress.q PRE-CREATION 
  ql/src/test/queries/clientpositive/vectorized_rcfile_columnar.q PRE-CREATION 
  ql/src/test/results/clientpositive/add_part_exist.q.out e64cc3c 
  ql/src/test/results/clientpositive/alter1.q.out a944b29 
  ql/src/test/results/clientpositive/alter2.q.out 82c78ad 
  ql/src/test/results/clientpositive/alter3.q.out 43064f7 
  ql/src/test/results/clientpositive/alter4.q.out ed8b4f6 
  ql/src/test/results/clientpositive/alter5.q.out c1df0c8 
  ql/src/test/results/clientpositive/alter_index.q.out a1738ad 
  ql/src/test/results/clientpositive/alter_rename_partition.q.out c6fa3f9 
  ql/src/test/results/clientpositive/describe_table_json.q.out d4f497f 
  ql/src/test/results/clientpositive/index_creation.q.out bddaa35 
  ql/src/test/results/clientpositive/input2.q.out e99d405 
  ql/src/test/results/clientpositive/input3.q.out be091ea 
  ql/src/test/results/clientpositive/input4.q.out 8717143 
  ql/src/test/results/clientpositive/plan_json.q.out b68dc06 
  ql/src/test/results/clientpositive/rename_column.q.out 34b70eb 
  ql/src/test/results/clientpositive/show_tables.q.out d434c0e 
  ql/src/test/results/clientpositive/vectorization_short_regress.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/vectorized_rcfile_columnar.q.out PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/columnar/ColumnarSerDe.java 70bc7e8 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java ec91717 

Diff: https://reviews.apache.org/r/14130/diff/


Testing
-------


Thanks,

Jitendra Pandey


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message