hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Kumar Singh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-7446) Add support to ALTER TABLE .. ADD COLUMN to Avro backed tables
Date Thu, 07 Aug 2014 21:39:12 GMT

    [ https://issues.apache.org/jira/browse/HIVE-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14089885#comment-14089885
] 

Ashish Kumar Singh commented on HIVE-7446:
------------------------------------------

[~szehon] thanks for your valuable insight here. This helped me discover that as of now alter
table is actually not working for avro backed tables in Hive. Alter table updates HMS with
new schema, but schema in the avro file containing avro-backed table's data keeps the original
schema. On trying to read from the avro-backed table after altering table, avro throws exception
while trying to read the avro file. This exception is thrown because of difference in expected
and actual schemas.

Based on offline discussion with [~tomwhite], Avro allows files written with the old schema
to be read with the new schema as long as certain rules are followed, e.g. adding a new field
has a default value. The full set of rules are at http://avro.apache.org/docs/current/spec.html#Schema+Resolution.

I will upload a patch that should fix this with appropriate tests.

> Add support to ALTER TABLE .. ADD COLUMN to Avro backed tables
> --------------------------------------------------------------
>
>                 Key: HIVE-7446
>                 URL: https://issues.apache.org/jira/browse/HIVE-7446
>             Project: Hive
>          Issue Type: New Feature
>            Reporter: Ashish Kumar Singh
>            Assignee: Ashish Kumar Singh
>         Attachments: HIVE-7446.patch
>
>
> HIVE-6806 adds native support for creating hive table stored as Avro. It would be good
to add support to ALTER TABLE .. ADD COLUMN to Avro backed tables.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message