hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3466) maintain dependency between external table partitions and managed table partitions
Date Fri, 14 Sep 2012 16:52:08 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455931#comment-13455931
] 

Namit Jain commented on HIVE-3466:
----------------------------------

Hive supports both partitioned and unpartitioned external tables. In both cases, when a new
table/partition is being added, the location is also specified for the new table/partition.
Let us consider a specific example:

create table T (key string, value string) partitioned by (ds string, hr string);
insert overwrite table T partition (ds='1', hr='1') ...;
..
insert overwrite table T partition (ds='1', hr='24') ...;

T is a partitioned table by date and hour, and Tsignal is an external table which conceptually
denotes the creation of the signal table.

create external table Tsignal (key string, value string) partitioned by (ds string);

When all the hourly partitions are created for a day (ds='1'), the corresponding partition
can be added to Tsignal

alter table Tsignal add partition (ds='1') location 'Location of T'/ds=1;

There is a implicit dependency between Tsignal@ds=1 and T@ds=1/hr=1, T@ds=1/hr=2, .... T@ds=1/hr=24,
but that dependency is not captured anywhere
in the metastore. It would be useful to have an ability to explicitly create that dependency.
This dependency can be used for all kinds of auditing purposes. For eg. when the following
query is performed:

select .. from Tsignal where ds = '1';

the inputs only contains Tsignal@ds=1, but is should also contain T@ds=1/hr=1, T@ds=1/hr=2,....T@ds=1/hr=24

The idea is to let the user specify and store the dependency explicitly.

alter table Tsignal add partition (ds='1') location 'Location of T'/ds=1 depends table T partition
(ds='1');
                
> maintain dependency between external table partitions and managed table partitions 
> -----------------------------------------------------------------------------------
>
>                 Key: HIVE-3466
>                 URL: https://issues.apache.org/jira/browse/HIVE-3466
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Namit Jain
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message