hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-401) SplitIntroducer does not handle some cases correctly
Date Wed, 27 Aug 2008 23:01:44 GMT
SplitIntroducer does not handle some cases correctly
----------------------------------------------------

                 Key: PIG-401
                 URL: https://issues.apache.org/jira/browse/PIG-401
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Pradeep Kamath
             Fix For: types_branch
         Attachments: PIG-401-partial.patch

The splitIntroducer does not handle the following cases correctly:
   * As soon as SplitIntroducer sees an operator which does NOT have multiple outputs it stops
- it should still recursively check the successors:
Current code which needs to change:
{code}
        List<LogicalOperator> sucs = mPlan.getSuccessors(root);
        if(sucs==null) return; 
        int size = sucs.size();
        if(size==0 || size==1) return;  --> THIS should change to recursively process the
succesor if it exists(size == 1 case)
{code}

   * If there are more than one operators in the plan which need implicit splits to be introduced
   * If the new SPLIT operator is introduced before a cogroup, the cogroup's inner map for
it's inputs and GroupBy expressions needs to be updated
   * The kind of changes need for cogroup maybe needed for some other operators 

I started looking at it some and have a patch for the first two issues - *UNTESTED* which
I am attaching to this issue

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message