hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariano Dominguez (JIRA)" <>
Subject [jira] [Updated] (HIVE-7046) Propagate addition of new columns to partition schema
Date Tue, 13 May 2014 18:46:18 GMT


Mariano Dominguez updated HIVE-7046:

    Affects Version/s: 0.11.0

> Propagate addition of new columns to partition schema
> -----------------------------------------------------
>                 Key: HIVE-7046
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Database/Schema
>    Affects Versions: 0.11.0, 0.12.0, 0.13.0
>            Reporter: Mariano Dominguez
> Hive reads data according to the partition schema, not the table schema (because of HIVE-3833).
ALTER TABLE only updates the table schema, and the changes are not propagated to partitions.
Thus, the schema of a partition will differ from that of the table after altering the table
schema; this is done to preserve the ability to read existing data, particularly when using
binary formats such as RCFile. Binary formats do not allow changing the type of a field because
of the way serialization works; a field serialized as a string will be displayed incorrectly
if read as an integer.
> Unfortunately, as a side effect, this behavior limits the ability to add new columns
to already exiting partitions using ALTER TABLE ADD COLUMNS. A possible workaround is to manually
recreate the partitions, but this process could be unnecessarily cumbersome if the number
of partitions is high. New columns should be propagated to existing partitions automatically

This message was sent by Atlassian JIRA

View raw message