hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pi Song (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-161) Rework physical plan
Date Tue, 15 Apr 2008 12:31:05 GMT

    [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589055#action_12589055
] 

Pi Song commented on PIG-161:
-----------------------------

Questions/comments :-

1) I spotted "new DefaultTuple();" in a few places. Shouldn't we use TupleFactory to create
tuples instead?
2) Why was the package named "topLevelOperators"? An abstract operator in any sense performs
the same thing regardless of where it is being used. I think we can categorize our operators
based on input/output types into three main groups: Relational(-Like), Aggregate, and Expression.
Some physical operators here can be used in local/MapReduce engine regardless of nesting level.
The dimensions we have to concern should be 1) Input/Output types  2) Generic or backend specific.
Adding an unuseful dimension will just introduce more complexity.

Unfortunately Java doesn't do higher dimensional packaging so the structure would look like
this:-
{noformat}
+PhysicalLayer
	+BackendNeutral
		+Relational
		+Aggregate
		+Expression
	+Hadoop
		+Relational
		+Aggregate
		+Expression
{noformat}

> Rework physical plan
> --------------------
>
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch, incr4.patch,
Phy_AbsClass.patch, pogenerate.patch, pogenerate.patch, pogenerate.patch
>
>
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message