hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas Nair" <the...@hortonworks.com>
Subject Re: Review Request 15663: Hive should be able to skip header and footer rows when reading data file for a table
Date Tue, 19 Nov 2013 04:08:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15663/#review29097
-----------------------------------------------------------



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
<https://reviews.apache.org/r/15663/#comment56238>

    config params also need to be added to hive-default.xml.template
    



ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java
<https://reviews.apache.org/r/15663/#comment56239>

    this code should be refactored to be re-used in both places.



ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java
<https://reviews.apache.org/r/15663/#comment56232>

    I think it is better to fail the query if it is not a number. Otherwise, user will silently
get data different from what is expected.
    



ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java
<https://reviews.apache.org/r/15663/#comment56233>

    need to update comment.
    same comment regarding exception handling as for header.
    



ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
<https://reviews.apache.org/r/15663/#comment56230>

    should this conf param go into HiveConf ?


- Thejas Nair


On Nov. 19, 2013, 1:31 a.m., Eric Hanson wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/15663/
> -----------------------------------------------------------
> 
> (Updated Nov. 19, 2013, 1:31 a.m.)
> 
> 
> Review request for hive and Thejas Nair.
> 
> 
> Bugs: HIVE-5795
>     https://issues.apache.org/jira/browse/HIVE-5795
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Hive should be able to skip header and footer rows when reading data file for a table
> 
> (I am uploading this on behalf of Shuaishuai Nie since he's not in the office)
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 32ab3d8 
>   data/files/header_footer_table_1/0001.txt PRE-CREATION 
>   data/files/header_footer_table_1/0002.txt PRE-CREATION 
>   data/files/header_footer_table_1/0003.txt PRE-CREATION 
>   data/files/header_footer_table_2/2012/01/01/0001.txt PRE-CREATION 
>   data/files/header_footer_table_2/2012/01/02/0002.txt PRE-CREATION 
>   data/files/header_footer_table_2/2012/01/03/0003.txt PRE-CREATION 
>   itests/qtest/pom.xml a453d8a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FetchOperator.java 5abcfc1 
>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java dd5cb6b

>   ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java 0ec6e63 
>   ql/src/test/org/apache/hadoop/hive/ql/io/TestHiveBinarySearchRecordReader.java 85dd975

>   ql/src/test/org/apache/hadoop/hive/ql/io/TestSymlinkTextInputFormat.java 0686d9b 
>   ql/src/test/queries/clientpositive/file_with_header_footer.q PRE-CREATION 
>   ql/src/test/results/clientpositive/file_with_header_footer.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/15663/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Eric Hanson
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message