hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Namit Jain (JIRA)" <>
Subject [jira] [Commented] (HIVE-933) Infer bucketing/sorting properties
Date Mon, 14 Jan 2013 03:54:12 GMT


Namit Jain commented on HIVE-933:

[~kevinwilfong], do you think we should differentiate between the fact that the user specified
bucketing vs. bucketing was inferred ?
For eg: merge cannot be performed on bucketed partitions. So, a working concatenate statement
today might start throwing an error if
this is deployed. The behavior should be different for the 2 scenarios: if bucketing/sorting
was inferred, merge should go through and
bucketing/sorting should be turned off.
> Infer bucketing/sorting properties
> ----------------------------------
>                 Key: HIVE-933
>                 URL:
>             Project: Hive
>          Issue Type: New Feature
>          Components: Query Processor
>            Reporter: Namit Jain
>            Assignee: Kevin Wilfong
>         Attachments: HIVE-933.1.patch.txt, HIVE-933.2.patch.txt, HIVE-933.3.patch.txt,
HIVE-933.4.patch.txt, HIVE-933.5.patch.txt, HIVE-933.6.patch.txt, HIVE-933.7.patch.txt, HIVE-933.8.patch.txt,
> This is a long-term plan, and may require major changes.
> From the query, we can figure out the sorting/bucketing properties, and change the metadata
of the destination at that time.
> However, this means that different partitions may have different metadata. Currently,
the query plan is same for all the 
> partitions of the table - we can do the following:
> 1. In the first cut, have a simple approach where you take the union all metadata, and
create the most defensive plan.
> 2. Enhance mapredWork() to include partition specific operator trees.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message