hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <>
Subject [jira] [Updated] (HIVE-6405) Support append feature for HCatalog
Date Thu, 20 Feb 2014 19:29:26 GMT


Sushanth Sowmyan updated HIVE-6405:

    Attachment:     (was: HIVE-6405.patch)

> Support append feature for HCatalog
> -----------------------------------
>                 Key: HIVE-6405
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Metastore, Query Processor, Thrift API
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
> HCatalog currently treats all tables as "immutable" - i.e. all tables and partitions
can be written to only once, and not appended. The nuances of what this means is as follows:
>  * A non-partitioned table can be written to, and data in it is never updated from then
on unless you drop and recreate.
>  * A partitioned table may support "appending" of a sort in a manner by adding new partitions
to the table, but once written, the partitions themselves cannot have any new data added to
> Hive, on the other hand, does allow us to "INSERT INTO" into a table, thus allowing us
append semantics. There is benefit to both of these models, and so, our goal is as follows:
> a) Introduce a notion of an immutable table, wherein all tables are not immutable by
default, and have this be a table property. If this property is set for a table, and we attempt
to write to a table that already has data (or a partition), disallow "INSERT INTO" into it
from hive. This property being set will allow hive to mimic HCatalog's current immutable-table
property. (I'm going to create a separate sub-task to cover this bit, and focus on the HCatalog-side
on this jira)
> b) As long as that flag is not set, HCatalog should be changed to allow appends into
it as well, and not simply error out if data already exists in a table.

This message was sent by Atlassian JIRA

View raw message