hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harish Butani (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-4108) Allow over() clause to contain an order by with no partition by
Date Thu, 07 Mar 2013 18:01:16 GMT

    [ https://issues.apache.org/jira/browse/HIVE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596119#comment-13596119
] 

Harish Butani commented on HIVE-4108:
-------------------------------------

A question I have is what should the behavior be when a Query level distribute and/or sort
is specified:
So for this query
{noformat}
select sum(x) over()
from t1
distribute by x
sort by y
{noformat}

Should the partition for sum be the entire table or be based on x. 
Today we support this query:
{noformat}
select sum(x)
from t1
distribute by x
sort by y
{noformat}
we infer the partition for sum to be x.

We also support this
{noformat}
select sum(x) over(row unbounded preceding and current row)
from t1
distribute by x
sort by y
{noformat}
again we infer the partition for sum to be x.

So:
- either we remove the concept of inferring from the Query level distribute/sort
- or a missing partition in an Over clause should imply the entire table only when there is
no Query level distribute/sort

Does this make sense?
                
> Allow over() clause to contain an order by with no partition by
> ---------------------------------------------------------------
>
>                 Key: HIVE-4108
>                 URL: https://issues.apache.org/jira/browse/HIVE-4108
>             Project: Hive
>          Issue Type: Bug
>          Components: PTF-Windowing
>            Reporter: Brock Noland
>
> HIVE-4073 allows over() to be called with no partition by and no order by. We should
allow only an order by.
> From the review of HIVE-4073:
> Ashutosh
> {noformat}
> Can you also add following test. This should also work.
> select p_name, p_retailprice,
> avg(p_retailprice) over(order by p_name)
> from part
> partition by p_name;
> {noformat}
> Harish
> {noformat}
> This test will not work (:
> The grammar needs to be changed so:
> partitioningSpec
> @init { msgs.push("partitioningSpec clause"); }
> @after { msgs.pop(); } 
> :
> partitionByClause orderByClause? -> ^(TOK_PARTITIONINGSPEC partitionByClause orderByClause?)
|
> orderByClause -> ^(TOK_PARTITIONINGSPEC orderByClause) |
> distributeByClause sortByClause? -> ^(TOK_PARTITIONINGSPEC distributeByClause sortByClause?)
|
> sortByClause? -> ^(TOK_PARTITIONINGSPEC sortByClause) |
> clusterByClause -> ^(TOK_PARTITIONINGSPEC clusterByClause)
> ;
> And the SemanticAnalyzer::processPTFPartitionSpec has to handle this shape of the AST
Tree. The PTFTranslator also needs changes. Do this as another Jira
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message