hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] Updated: (PIG-1321) Logical Optimizer: Merge cascading foreach
Date Fri, 27 Aug 2010 22:25:55 GMT

     [ https://issues.apache.org/jira/browse/PIG-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-1321:
----------------------------

    Attachment: jira-1321-3.patch

Repost the pre-condition:
1. two consecutive foreach statements.
2. the second foreach statement is a simple inner plan in which the ognly statement is a GENERATE
statement. In other words, the second foreach statement must be something like "FOREACH A
GENERATE ...."
3. The first foreach statement cannot contain flatten due to its complexity
4. No 1st foreach output is referred more than once in second foreach, eg: "B = foreach xxxx;
C = foreach B generate $0, $1, $0" will not be merged. The reason if we merge, $0 will be
calculated twice, which defeat the benefit of merging.

All tests pass. test-patch result:
     [exec] +1 overall.  
     [exec] 
     [exec]     +1 @author.  The patch does not contain any @author tags.
     [exec] 
     [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
     [exec] 
     [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
     [exec] 
     [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler
warnings.
     [exec] 
     [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
     [exec] 
     [exec]     +1 release audit.  The applied patch does not increase the total number of
release audit warnings.

> Logical Optimizer: Merge cascading foreach
> ------------------------------------------
>
>                 Key: PIG-1321
>                 URL: https://issues.apache.org/jira/browse/PIG-1321
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>    Affects Versions: 0.7.0
>            Reporter: Daniel Dai
>            Assignee: Xuefu Zhang
>             Fix For: 0.8.0
>
>         Attachments: jira-1321-2.patch, jira-1321-3.patch, pig-1321.patch
>
>
> We can merge consecutive foreach statement.
> Eg:
> b = foreach a generate a0#'key1' as b0, a0#'key2' as b1, a1;
> c = foreach b generate b0#'kk1', b0#'kk2', b1, a1;
> => c = foreach a generate a0#'key1'#'kk1', a0#'key1'#'kk2', a0#'key2', a1;

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message