hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] Created: (HIVE-1781) outputs not populated for dynamic partitions at compile time
Date Wed, 10 Nov 2010 18:05:13 GMT
outputs not populated for dynamic partitions at compile time

                 Key: HIVE-1781
             Project: Hive
          Issue Type: Bug
            Reporter: Namit Jain

OSTHOOK: query: create table tstsrcpart like srcpart
POSTHOOK: Output: default@tstsrcpart
PREHOOK: query: from srcpart
insert overwrite table tstsrcpart partition (ds, hr) select key, value, ds, hr where ds <=
PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=11
PREHOOK: Input: default@srcpart@ds=2008-04-08/hr=12
POSTHOOK: query: from srcpart

As is evident from above, the outputs are not populated at all at compile time.

This may create a problem for many components that depend on outputs: locking, authorization
However, the exact set of outputs may be needed for some other components (for example. the
internal deployment in Facebook has a replication hook which is used for replication which
needs the
exact set of outputs). It may  be a good idea to extend WriteEntity to include a flag which
whether the output is complete or not, and then the hook can look at that flag if needed

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message