hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21667) Bad error message when non-ACID files are put in an insert_only ACID table
Date Thu, 23 May 2019 07:17:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16846505#comment-16846505
] 

Todd Lipcon commented on HIVE-21667:
------------------------------------

[~vgumashta] any thoughts on this? I was under the impression that a post-upgrade ACID table
would have files lying around in the old non-ACID layout, because we don't want to incur a
bunch of 'move' operations during conversion from non-ACID to ACID. Am I wrong about that?
In Impala's implementation, should we bail if we see a non-conforming file in an ACID table
directory?

> Bad error message when non-ACID files are put in an insert_only ACID table
> --------------------------------------------------------------------------
>
>                 Key: HIVE-21667
>                 URL: https://issues.apache.org/jira/browse/HIVE-21667
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 3.1.0
>            Reporter: Todd Lipcon
>            Priority: Major
>
> I created an insert_only transactional table, and then dropped a text file into the table's
directory from a non-transaction-aware client. When I next queried the table, I got the following
error:
> Error: java.io.IOException: java.io.IOException: Not a file: hdfs://localhost:20500/test-warehouse/trans/delta_0000002_0000002_0000
(state=,code=0)
> It seems that Hive saw the non-ACID file and fell back to some kind of non-ACID reader
path, but then got confused by the delta directory. This case should either fall back to gracefully
reading the file, or give an error message like "Unexpected file not conforming to ACID layout:
<path>. Data must be loaded using into transactional tables LOAD DATA." or something.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message