hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1644) New logical plan: Plan.connect with position is misused in some places
Date Thu, 23 Sep 2010 18:00:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-1644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12914130#action_12914130
] 

Thejas M Nair commented on PIG-1644:
------------------------------------

These operations will be fairly common in the optimizer. I think it would be good to have
functions in the OperatorPlan that support these operations, that will reduce the chances
of bugs and also make the code more readable.


> New logical plan: Plan.connect with position is misused in some places
> ----------------------------------------------------------------------
>
>                 Key: PIG-1644
>                 URL: https://issues.apache.org/jira/browse/PIG-1644
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.8.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.8.0
>
>         Attachments: PIG-1644-1.patch
>
>
> When we replace/remove/insert a node, we will use disconnect/connect methods of OperatorPlan.
When we disconnect an edge, we shall save the position of the edge in origination and destination,
and use this position when connect to the new predecessor/successor. Some of the pattens are:
> Insert a new node:
> {code}
> Pair<Integer, Integer> pos = plan.disconnect(pred, succ);
> plan.connect(pred, pos.first, newnode, 0);
> plan.connect(newnode, 0, succ, pos.second);
> {code}
> Remove a node:
> {code}
> Pair<Integer, Integer> pos1 = plan.disconnect(pred, nodeToRemove);
> Pair<Integer, Integer> pos2 = plan.disconnect(nodeToRemove, succ);
> plan.connect(pred, pos1.first, succ, pos2.second);
> {code}
> Replace a node:
> {code}
> Pair<Integer, Integer> pos1 = plan.disconnect(pred, nodeToReplace);
> Pair<Integer, Integer> pos2 = plan.disconnect(nodeToReplace, succ);
> plan.connect(pred, pos1.first, newNode, pos1.second);
> plan.connect(newNode, pos2.first, succ, pos2.second);
> {code}
> There are couple of places of we does not follow this pattern, that results some error.
For example, the following script fail:
> {code}
> a = load '1.txt' as (a0, a1, a2, a3);
> b = foreach a generate a0, a1, a2;
> store b into 'aaa';
> c = order b by a2;
> d = foreach c generate a2;
> store d into 'bbb';
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message