hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-4041) Support multiple partitionings in a single Query
Date Wed, 13 Mar 2013 20:00:14 GMT

     [ https://issues.apache.org/jira/browse/HIVE-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-4041:
------------------------------

    Attachment: HIVE-4041.D9381.1.patch

hbutani requested code review of "HIVE-4041 [jira] Support multiple partitionings in a single
Query".

Reviewers: JIRA, ashutoshc

add support for multiple partitioning

Currently we disallow queries if the partition specifications of all Wdw fns are not the same.
We can relax this by generating multiple PTFOps based on the unique partitionings in a Query.
For partitionings that only differ in sort, we can introduce a sort step in between PTFOps,
which can happen in the same Reduce task.

TEST PLAN
  added tests to windowing_multipartitioning.q

REVISION DETAIL
  https://reviews.facebook.net/D9381

AFFECTED FILES
  data/files/flights_tiny.txt
  data/files/part.rc
  data/files/part.seq
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFInvocationSpec.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingComponentizer.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/WindowingSpec.java
  ql/src/test/queries/clientpositive/windowing_multipartitioning.q
  ql/src/test/results/clientpositive/windowing_multipartitioning.q.out

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/22545/

To: JIRA, ashutoshc, hbutani

                
> Support multiple partitionings in a single Query
> ------------------------------------------------
>
>                 Key: HIVE-4041
>                 URL: https://issues.apache.org/jira/browse/HIVE-4041
>             Project: Hive
>          Issue Type: Bug
>          Components: PTF-Windowing
>            Reporter: Harish Butani
>            Assignee: Harish Butani
>         Attachments: HIVE-4041.D9381.1.patch, WindowingComponentization.pdf
>
>
> Currently we disallow queries if the partition specifications of all Wdw fns are not
the same. We can relax this by generating multiple PTFOps based on the unique partitionings
in a Query. For partitionings that only differ in sort, we can introduce a sort step in between
PTFOps, which can happen in the same Reduce task.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message