hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland" <br...@cloudera.com>
Subject Re: Review Request 17061: HIVE-5783 - Native Parquet Support in Hive
Date Tue, 04 Feb 2014 14:20:23 GMT


> On Feb. 3, 2014, 11:53 p.m., Xuefu Zhang wrote:
> > ql/src/java/parquet/hive/DeprecatedParquetInputFormat.java, line 22
> > <https://reviews.apache.org/r/17061/diff/5/?file=454363#file454363line22>
> >
> >     I don't quite get how this provides backward compatibility. Please explain.

The issue is that users til now specified the create table statement below which puts those
class names in the metastore. The same is true for all parquet.hive.* classes included in
this change.

CREATE TABLE parquet_test (column1 string)
ROW FORMAT SERDE 'parquet.hive.serde.ParquetHiveSerDe'
  STORED AS
   INPUTFORMAT 'parquet.hive.DeprecatedParquetInputFormat'
   OUTPUTFORMAT 'parquet.hive.DeprecatedParquetOutputFormat';


- Brock


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17061/#review33473
-----------------------------------------------------------


On Jan. 30, 2014, 2:48 p.m., Brock Noland wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17061/
> -----------------------------------------------------------
> 
> (Updated Jan. 30, 2014, 2:48 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-5783
>     https://issues.apache.org/jira/browse/HIVE-5783
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Adds native parquet support hive
> 
> 
> Diffs
> -----
> 
>   data/files/parquet_create.txt PRE-CREATION 
>   data/files/parquet_partitioned.txt PRE-CREATION 
>   pom.xml 41f5337 
>   ql/pom.xml 7087a4c 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetInputFormat.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ProjectionPusher.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ArrayWritableGroupConverter.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableGroupConverter.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/DataWritableRecordConverter.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveGroupConverter.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/HiveSchemaConverter.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/DataWritableReadSupport.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/AbstractParquetMapInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/DeepParquetHiveMapInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveArrayInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/StandardParquetHiveMapInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetByteInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetPrimitiveInspectorFactory.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetShortInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/primitive/ParquetStringInspector.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/writable/BigDecimalWritable.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/writable/BinaryWritable.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriteSupport.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/DataWritableWriter.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 13d0a56 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g f83c15d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 010e04f 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 538b2b0 
>   ql/src/java/parquet/hive/DeprecatedParquetInputFormat.java PRE-CREATION 
>   ql/src/java/parquet/hive/DeprecatedParquetOutputFormat.java PRE-CREATION 
>   ql/src/java/parquet/hive/MapredParquetInputFormat.java PRE-CREATION 
>   ql/src/java/parquet/hive/MapredParquetOutputFormat.java PRE-CREATION 
>   ql/src/java/parquet/hive/serde/ParquetHiveSerDe.java PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestHiveSchemaConverter.java PRE-CREATION

>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestMapredParquetInputFormat.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestMapredParquetOutputFormat.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetSerDe.java PRE-CREATION

>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestAbstractParquetMapInspector.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestDeepParquetHiveMapInspector.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetHiveArrayInspector.java
PRE-CREATION 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestStandardParquetHiveMapInspector.java
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_create.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_partitioned.q PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_create.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_partitioned.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/17061/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Brock Noland
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message