hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pi Song (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-161) Rework physical plan
Date Thu, 17 Apr 2008 14:55:23 GMT

    [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12590048#action_12590048
] 

Pi Song commented on PIG-161:
-----------------------------

I've been thinking about the work distribution layer a bit more. I still  have no idea how
this thing could be organized as a layer. In a naive way, we could compile PO layer to a new
graph in a new layer but that would introduce another unnecessary translation plus some more
dummy POs in the current PO layer. 

One thing I realized is that only a few operators exhibit properties of work distribution
constructs - LOAD (load + parallelizer), STORE ( stream Aggregator + store), and Global Rearranger
( stream aggregator + actual global rearranger logic + parallelizer). These are only operators
to be implemented specific to the backend. So, why don't we just have these operators as backend
specific operators.
{noformat}
	+PhysicalOperator
		+Neutral
		+HadoopSpecific
		+LocalEngineSpecific
{noformat}

@Alan: I will help you think about EvalFunc as soon as possible.

> Rework physical plan
> --------------------
>
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch, incr4.patch,
Phy_AbsClass.patch, pogenerate.patch, pogenerate.patch, pogenerate.patch
>
>
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message