hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-13974) ORC Schema Evolution doesn't support add columns to non-last STRUCT columns
Date Mon, 11 Jul 2016 18:46:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15371388#comment-15371388
] 

Owen O'Malley commented on HIVE-13974:
--------------------------------------

First pass comments on the ORC changes:

* You *must* include unit tests in the ORC module for changes there.
* Don't move checkAcidSchema around and certainly don't make it a public API. We should probably
have ReaderImpl pass a boolean to the constructor of SchemaEvolution saying that the file
is Acid. Using the column names is bad and we should probably move over to use the acid stats
property as the check.
* SameCategoryAndAttributes is a duplication of TypeDescription.equals.
* We need to integrate this with ORC-54 too.
* I like pulling the include logic into SchemaEvolution.
* Please use 'reader' instead of 'logical' in the names in SchemaEvolution.

I'm still going through the SchemaEvolution changes.

> ORC Schema Evolution doesn't support add columns to non-last STRUCT columns
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-13974
>                 URL: https://issues.apache.org/jira/browse/HIVE-13974
>             Project: Hive
>          Issue Type: Bug
>          Components: Hive, ORC, Transactions
>    Affects Versions: 1.3.0, 2.1.0, 2.2.0
>            Reporter: Matt McCline
>            Assignee: Matt McCline
>            Priority: Blocker
>         Attachments: HIVE-13974.01.patch, HIVE-13974.02.patch, HIVE-13974.03.patch, HIVE-13974.04.patch,
HIVE-13974.05.WIP.patch, HIVE-13974.06.patch, HIVE-13974.07.patch, HIVE-13974.08.patch, HIVE-13974.09.patch,
HIVE-13974.091.patch
>
>
> Currently, the included columns are based on the fileSchema and not the readerSchema
which doesn't work for adding columns to non-last STRUCT data type columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message