pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pi Song (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-158) Rework logical plan
Date Tue, 01 Apr 2008 12:38:24 GMT

    [ https://issues.apache.org/jira/browse/PIG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12584120#action_12584120

Pi Song commented on PIG-158:

This is not good for nested operations.

I really want to see we switch to the plans in plans approach. This way those operators that
do not have predecessors aren't problems because they are not root nodes on the main logical

Take LOFilter as example. This operator takes a bag of tuples as input in the primary plan.
For each input tuple, there is a transient internal logical plan that does consume the tuple
and outputs something. Then the output tuple gets processed in the outer operator.

We can have special load/store that only work with one tuple in a bag at a time to make this
more clean.

Visitors when visiting a node can visit internal nodes in internal plans first before visiting
the actual node. We may need to have a distinction between operators that have plans and operators
that don't. This is not too complex in my opinion.

The nested data model in Pig is already good. I just want to see nested plans to cope with
it nicely.

> Rework logical plan
> -------------------
>                 Key: PIG-158
>                 URL: https://issues.apache.org/jira/browse/PIG-158
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: logical_operators.patch, logical_operators_rev_1.patch, logical_operators_rev_2.patch,
> Rework the logical plan in line with http://wiki.apache.org/pig/PigExecutionModel

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message