hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Created: (PIG-520) Physical plan cloning could lead to out of order connections
Date Mon, 10 Nov 2008 21:01:44 GMT
Physical plan cloning could lead to out of order connections

                 Key: PIG-520
                 URL: https://issues.apache.org/jira/browse/PIG-520
             Project: Pig
          Issue Type: Bug
    Affects Versions: types_branch
            Reporter: Santhosh Srinivasan
             Fix For: types_branch

In the PhysicalPlan clone method, the algorithm used is as follows:

1. Create an empty plan

2. For all the operators in the plan, 
   a. clone the operator 
   b. add it to the plan

3. For all the keys (from_node) in the map mFromEdges
   a. For all the values (to_node) for this key
      i. Connect the from_node to the to_node in the plan

There are no guarantees on the order in which the from_nodes in the mFromEdges are processed,
we could get out of order connections in the graph.


If we have UDF with two arguments like myUDF(a, b) in a plan, the order in which the nodes
are processed will determine the cloned plan. We could end up with 

myUDF(a, b)


myUDF(b,. a)

depending on the order in which a and b appear in the mFromEdges look up table.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message