hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-922) Logical optimizer: push up project
Date Tue, 25 Aug 2009 17:55:59 GMT

    [ https://issues.apache.org/jira/browse/PIG-922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747521#action_12747521
] 

Santhosh Srinivasan commented on PIG-922:
-----------------------------------------

I am not sure about the logic for handling cogroup. Let me take another example of an operator
with multiple inputs - union. If you look at the code below, the method returns a single required
fields element. The required fields element contains a reference to all the inputs that are
required to compute that particular column. However, wrt cogroup you are returning a list
of required fields that contains nulls for all the positions that are of no interest.

{code}
+        ArrayList<Pair<Integer, Integer>> inputList = new ArrayList<Pair<Integer,
Integer>>();
+        for (int i=0;i<predecessors.size();i++)
+            inputList.add(new Pair<Integer, Integer>(i, column));
+        List<RequiredFields> result = new ArrayList<RequiredFields>();
+        result.add(new RequiredFields(inputList));
+        return result;
{code}

> Logical optimizer: push up project
> ----------------------------------
>
>                 Key: PIG-922
>                 URL: https://issues.apache.org/jira/browse/PIG-922
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>    Affects Versions: 0.3.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.4.0
>
>         Attachments: PIG-922-p1_0.patch, PIG-922-p1_1.patch, PIG-922-p1_2.patch
>
>
> This is a continuation work of [PIG-697|https://issues.apache.org/jira/browse/PIG-697].
We need to add another rule to the logical optimizer: Push up project, ie, prune columns as
early as possible.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message