hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Singh" <asi...@cloudera.com>
Subject Re: Review Request 28372: Address review comments
Date Fri, 17 Apr 2015 03:02:39 GMT


> On Dec. 13, 2014, 1:20 a.m., Ryan Blue wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java, line
138
> > <https://reviews.apache.org/r/28372/diff/8/?file=790874#file790874line138>
> >
> >     I'd rather not use parquet.file at all. Is there a compelling reason to keep
it now that this can find files in the table location?
> 
> Ashish Singh wrote:
>     parquet.file or a similar table property will be required to create a parquet backed
managed table. Hive expects atleast one column in the table, which can either come from user
specified hive schema or from table's serde.

Removed parquet.file and restricted the automatic schema generation to external tables poiniting
to dir with parquet data.


- Ashish


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/28372/#review65004
-----------------------------------------------------------


On April 17, 2015, 3:01 a.m., Ashish Singh wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/28372/
> -----------------------------------------------------------
> 
> (Updated April 17, 2015, 3:01 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-8950
>     https://issues.apache.org/jira/browse/HIVE-8950
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-8950: Add support in ParquetHiveSerde to create table schema from a parquet file
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java e138800e6dadd6fe76345f21eb76c906165c438d

>   hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java 53d913435cd43c96f044cf8668461fc686817ef4

>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 3f267ff0eb20560c36a19b74353f9d6749c8b333

>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 907ad2b8d7bae9b13eb4d9605ff2a3fe60c03ee8

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ParquetSchemaReader.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ParquetToHiveSchemaConverter.java
PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java 7fd5e9612d4e3c9bf3b816bc48dbdbe59fb8a5a8

>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java a029f10c116fd46070b2d41790043f0a7001390f

>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetToHiveSchemaConverter.java
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_multi_field_struct_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_optional_elements_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_required_elements_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_single_field_struct_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_structs_gen_schema.q PRE-CREATION

>   ql/src/test/queries/clientpositive/parquet_array_of_structs_gen_schema_ext.q PRE-CREATION

>   ql/src/test/queries/clientpositive/parquet_array_of_unannotated_groups_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_array_of_unannotated_primitives_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_avro_array_of_primitives_gen_schema.q PRE-CREATION

>   ql/src/test/queries/clientpositive/parquet_avro_array_of_single_field_struct_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_decimal_gen_schema.q PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_thrift_array_of_primitives_gen_schema.q
PRE-CREATION 
>   ql/src/test/queries/clientpositive/parquet_thrift_array_of_single_field_struct_gen_schema.q
PRE-CREATION 
>   ql/src/test/results/clientpositive/create_view_partitioned.q.out ebf9a6bc4f2321d7f539b7a445b3f279e3285b8a

>   ql/src/test/results/clientpositive/parquet_array_of_multi_field_struct_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_array_of_optional_elements_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_array_of_required_elements_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_array_of_single_field_struct_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_array_of_structs_gen_schema.q.out PRE-CREATION

>   ql/src/test/results/clientpositive/parquet_array_of_structs_gen_schema_ext.q.out PRE-CREATION

>   ql/src/test/results/clientpositive/parquet_array_of_unannotated_groups_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_array_of_unannotated_primitives_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_avro_array_of_primitives_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_avro_array_of_single_field_struct_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_decimal_gen_schema.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_thrift_array_of_primitives_gen_schema.q.out
PRE-CREATION 
>   ql/src/test/results/clientpositive/parquet_thrift_array_of_single_field_struct_gen_schema.q.out
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/28372/diff/
> 
> 
> Testing
> -------
> 
> Tested by adding appropriate qTests.
> 
> 
> Thanks,
> 
> Ashish Singh
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message