hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuefu Zhang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1574) Optimization rule PushUpFilter causes filter to be pushed up out joins
Date Fri, 27 Aug 2010 19:04:53 GMT

     [ https://issues.apache.org/jira/browse/PIG-1574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xuefu Zhang updated PIG-1574:
-----------------------------

    Attachment:     (was: jira-1574-1.patch)

> Optimization rule PushUpFilter causes filter to be pushed up out joins
> ----------------------------------------------------------------------
>
>                 Key: PIG-1574
>                 URL: https://issues.apache.org/jira/browse/PIG-1574
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>             Fix For: 0.8.0
>
>
> The PushUpFilter optimization rule in the new logical plan moves the filter up to one
of the join branch. It does this aggressively by find an operator that has all the projection
UIDs. However, it didn't consider that the found operator might be another join. If that join
is outer, then we cannot simply move the filter to one of its branches.
> As an example, the following script will be erroneously optimized:
>         A = load 'myfile' as (d1:int);
>         B = load 'anotherfile' as (d2:int);
>         C = join A by d1 full outer, B by d2;        
>         D = load 'xxx' as (d3:int);
>         E = join C by d1, D by d3;        
>         F = filter E by d1 > 5;
>         G = store F into 'dummy';

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message