hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-13185) orc.ReaderImp.ensureOrcFooter() method fails on small text files with IndexOutOfBoundsException
Date Mon, 29 Feb 2016 23:08:18 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15172816#comment-15172816
] 

Sergey Shelukhin edited comment on HIVE-13185 at 2/29/16 11:07 PM:
-------------------------------------------------------------------

Both can be done. My point is that the exceptions from corrupt files are often expected, so
if there's some other issue like this, it's better to have it caught properly in the validator,
to make sure we recognize the file as invalid instead of failing. Note that there isn't even
an ORC table in the example, it's just running the validator on all supported formats to see
if it happens to be ORC/...


was (Author: sershe):
Both can be done. My point is that the exceptions from corrupt files are often expected, so
if there's some other issue like this, it's better to have it caught properly in the validator,
to make sure we recognize the file as invalid instead of failing.

> orc.ReaderImp.ensureOrcFooter() method fails on small text files with IndexOutOfBoundsException
> -----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-13185
>                 URL: https://issues.apache.org/jira/browse/HIVE-13185
>             Project: Hive
>          Issue Type: Bug
>          Components: ORC
>    Affects Versions: 2.1.0
>            Reporter: Illya Yalovyy
>
> Steps to reproduce:
> 1. Create a Text source table with one line of data:
> {code}
> create table src (id int);
> insert overwrite table src values (1);
> {code}
> 2. Create a target table:
> {code}
> create table trg (id int);
> {code}
> 3. Try to load small text file to the target table:
> {code}
> load data inpath 'user/hive/warehouse/src/000000_0' into table trg;
> {code}
> *Error message:*
> {quote}
> FAILED: SemanticException Unable to load data to destination table. Error: java.lang.IndexOutOfBoundsException
> {quote}
> *Stack trace:*
> {noformat}
> org.apache.hadoop.hive.ql.parse.SemanticException: Unable to load data to destination
table. Error: java.lang.IndexOutOfBoundsException
> 	at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.ensureFileFormatsMatch(LoadSemanticAnalyzer.java:340)
> 	at org.apache.hadoop.hive.ql.parse.LoadSemanticAnalyzer.analyzeInternal(LoadSemanticAnalyzer.java:224)
> 	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:242)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:481)
> 	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:317)
> 	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1190)
> 	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1285)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1116)
> 	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1104)
> ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message