hadoop-hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HIVE-788) Triggers when a new partition is created for a table
Date Tue, 25 Aug 2009 00:08:59 GMT

    [ https://issues.apache.org/jira/browse/HIVE-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747151#action_12747151

Zheng Shao commented on HIVE-788:

For A, we can add a command to Hive to return a partition with partition keys greater than
a given partition. Something like "wait for next partition of table xxx [greater than ds=yyy,
ts=zzz]". This command will return the first partition after the given partition. If the wait
command is implemented natively using hive, the cost of each probe will be just a SQL query
on the metastore (so the probe interval can be something like one minute or less).

Once we get the new partition, the client can decide what to do. This can avoid hard questions
like B.1, B.2, B.3.

> Triggers when a new partition is created for a table
> ----------------------------------------------------
>                 Key: HIVE-788
>                 URL: https://issues.apache.org/jira/browse/HIVE-788
>             Project: Hadoop Hive
>          Issue Type: New Feature
>            Reporter: Zheng Shao
> One requirement for HIVE-787 is that users would like to run a command whenever a new
partition of a Hive table gets created.
> There are several ways to achieve this functionality:
> A. Probe and wait: We can have the scripts running in a loop checking if a new partition
is created.
>   Pros: easy to write, easy to control
>   Cons: will introduce another delay based on the probing interval.
> B. Triggered: The command is registered inside the hive metastore. Whenever a partition
gets created, we run the registered command. 
> Several questions around option B are:
> 1. whether to support registration of HiveQL or shell command;
> 2. which machine/environment to run the command;
> 3. what to do if the registered command failed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message