hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sushanth Sowmyan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-6405) Support append feature for HCatalog
Date Wed, 19 Feb 2014 17:14:23 GMT

     [ https://issues.apache.org/jira/browse/HIVE-6405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sushanth Sowmyan updated HIVE-6405:
-----------------------------------

    Status: Patch Available  (was: Open)

> Support append feature for HCatalog
> -----------------------------------
>
>                 Key: HIVE-6405
>                 URL: https://issues.apache.org/jira/browse/HIVE-6405
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog, Metastore, Query Processor, Thrift API
>            Reporter: Sushanth Sowmyan
>            Assignee: Sushanth Sowmyan
>         Attachments: HIVE-6405.patch
>
>
> HCatalog currently treats all tables as "immutable" - i.e. all tables and partitions
can be written to only once, and not appended. The nuances of what this means is as follows:
>  * A non-partitioned table can be written to, and data in it is never updated from then
on unless you drop and recreate.
>  * A partitioned table may support "appending" of a sort in a manner by adding new partitions
to the table, but once written, the partitions themselves cannot have any new data added to
them.
> Hive, on the other hand, does allow us to "INSERT INTO" into a table, thus allowing us
append semantics. There is benefit to both of these models, and so, our goal is as follows:
> a) Introduce a notion of an immutable table, wherein all tables are not immutable by
default, and have this be a table property. If this property is set for a table, and we attempt
to write to a table that already has data (or a partition), disallow "INSERT INTO" into it
from hive. This property being set will allow hive to mimic HCatalog's current immutable-table
property. (I'm going to create a separate sub-task to cover this bit, and focus on the HCatalog-side
on this jira)
> b) As long as that flag is not set, HCatalog should be changed to allow appends into
it as well, and not simply error out if data already exists in a table.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message