hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-4358) Check for Map side processing in PTFOp is no longer valid
Date Mon, 22 Apr 2013 22:29:18 GMT

     [ https://issues.apache.org/jira/browse/HIVE-4358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HIVE-4358:
------------------------------

    Attachment: HIVE-4358.D10473.1.patch

hbutani requested code review of "HIVE-4358 [jira] Check for Map side processing in PTFOp
is no longer valid".

Reviewers: JIRA, ashutoshc

add isMapFlag to PTFDesc

With the changes for ReduceSinkDedup it is no longer true that a non Map-side PTF Operator
is preceded by an ExtractOp. For e.g. following query can produce the issue:

create view IF NOT EXISTS mfgr_price_view as
select p_mfgr, p_brand,
sum(p_retailprice) as s
from part
group by p_mfgr, p_brand;

select p_mfgr, p_brand, s,
sum(s) over w1  as s1
from mfgr_price_view
window w1 as (distribute by p_mfgr sort by p_brand rows between 2 preceding and current row);

Fix is to add an explicit flag to PTFDesc

TEST PLAN
  add test to windowing.q

REVISION DETAIL
  https://reviews.facebook.net/D10473

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/PTFOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/plan/PTFDesc.java
  ql/src/test/queries/clientpositive/windowing.q
  ql/src/test/results/clientpositive/windowing.q.out

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/25035/

To: JIRA, ashutoshc, hbutani

                
> Check for Map side processing in PTFOp is no longer valid
> ---------------------------------------------------------
>
>                 Key: HIVE-4358
>                 URL: https://issues.apache.org/jira/browse/HIVE-4358
>             Project: Hive
>          Issue Type: Bug
>          Components: PTF-Windowing
>            Reporter: Harish Butani
>         Attachments: HIVE-4358.D10473.1.patch
>
>
> With the changes for ReduceSinkDedup it is no longer true that a non Map-side PTF Operator
is preceded by an ExtractOp. For e.g. following query can produce the issue:
> {noformat}
> create view IF NOT EXISTS mfgr_price_view as 
> select p_mfgr, p_brand, 
> sum(p_retailprice) as s 
> from part 
> group by p_mfgr, p_brand;
>         
> select p_mfgr, p_brand, s, 
> sum(s) over w1  as s1
> from mfgr_price_view 
> window w1 as (distribute by p_mfgr sort by p_brand rows between 2 preceding and current
row);
> {noformat}
> Fix is to add an explicit flag to PTFDesc

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message