hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "cheng xu" <cheng.a...@intel.com>
Subject Re: Review Request 34473: HIVE-10749 Implement Insert statement for parquet
Date Fri, 22 May 2015 06:27:11 GMT


> On May 21, 2015, 7:18 p.m., Sergio Pena wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java,
line 59
> > <https://reviews.apache.org/r/34473/diff/2/?file=966160#file966160line59>
> >
> >     Could you separate words with _? Like ENABLE_ACID_SCHEMA_INFO. It helps to read
the constant more easily.
> >     
> >     Do we have to enable transactions exclusively for parquet? Isn't there another
variable that enables trasnactions on Hive that we can use?

This variable is used for setting the schema for parquet. It's only related to whether you
need to write data to base file or not. So we have to use this way to append the original
data with ACID info.


> On May 21, 2015, 7:18 p.m., Sergio Pena wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java,
lines 98-103
> > <https://reviews.apache.org/r/34473/diff/2/?file=966160#file966160line98>
> >
> >     You can use this one line to return the column list:
> >     
> >     return (List<String>) StringUtils.getStringCollection(tableProperties.getProperty(IOConstants.COLUMNS));
> >     
> >     It will return an empty list array if COLUMN is empty.

Great suggestion!


- cheng


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34473/#review84758
-----------------------------------------------------------


On May 22, 2015, 6:26 a.m., cheng xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34473/
> -----------------------------------------------------------
> 
> (Updated May 22, 2015, 6:26 a.m.)
> 
> 
> Review request for hive, Alan Gates, Owen O'Malley, and Sergio Pena.
> 
> 
> Bugs: HIVE-10749
>     https://issues.apache.org/jira/browse/HIVE-10749
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Implement the insert statement for parquet format.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/MapredParquetOutputFormat.java c6fb26c

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/acid/ParquetRecordUpdater.java PRE-CREATION

>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/read/ParquetRecordReaderWrapper.java
f513572 
>   ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ObjectArrayWritableObjectInspector.java
571f993 
>   ql/src/test/org/apache/hadoop/hive/ql/io/parquet/acid/TestParquetRecordUpdater.java
PRE-CREATION 
>   ql/src/test/queries/clientpositive/acid_parquet_insert.q PRE-CREATION 
>   ql/src/test/results/clientpositive/acid_parquet_insert.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34473/diff/
> 
> 
> Testing
> -------
> 
> Newly added qtest and UT passed locally
> 
> 
> Thanks,
> 
> cheng xu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message