hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-158) Rework logical plan
Date Tue, 10 Jun 2008 01:01:46 GMT

    [ https://issues.apache.org/jira/browse/PIG-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12603746#action_12603746
] 

Santhosh Srinivasan commented on PIG-158:
-----------------------------------------

"Re: Eliminating the Generate Operator": Translating the Generate Operator to the Accumulator

Instead of eliminating the Generate Operator, we can instead translate the generate operator
to the accumulate operator or overload the POGenerate to provide the ability to accumulate
tuples. The logical plan will remain unchanged. The logical to physical translator will do
the following:

In order to translate the Generate operator, the nested plans which are currently part of
the Generate will be promoted to be a part of the Foreach operator with the following changes:

1. In the nested plan of Foreach remove any leaves that are not Generate

2. Any expression that is part of the generate (root of the nested plan) which does not require
Generate's input will be moved into a nested plan of Foreach.

3. The remaining expressions of generate will be attached to Generate by duplicating the graph.

4. The physical translation of Generate will now be the new Accumulate Operator or POGenerate
with the added functionality of being able to accumulate.

> Rework logical plan
> -------------------
>
>                 Key: PIG-158
>                 URL: https://issues.apache.org/jira/browse/PIG-158
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: is_null.patch, logical_operators.patch, logical_operators_rev_1.patch,
logical_operators_rev_2.patch, logical_operators_rev_3.patch, parser_changes.patch, parser_changes_v1.patch,
parser_changes_v2.patch, parser_changes_v3.patch, parser_changes_v4.patch, ParserErrors.txt,
udf_fix.patch, udf_funcSpec.patch, udf_return_type.patch, user_func_and_store.patch, visitorWalker.patch
>
>
> Rework the logical plan in line with http://wiki.apache.org/pig/PigExecutionModel

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message