hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-161) Rework physical plan
Date Fri, 25 Apr 2008 15:45:57 GMT

    [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12592411#action_12592411

Shravan Matthur Narayanamurthy commented on PIG-161:

Looks good Shubham. Just a few more clarifications. 
@Alan: Some of these require comments from you

1) As the POSort & PODistinct have an Iterator & I checked that the DataBag iterators
do not implement serializable, I think you should mark them transient and handle it appropriately.

2) Since POUserFuncs only occur in nested plans, I was wondering about the return type for
them. Because, POGenerate uses the return type to decide what getNext method to call, it would
fail with the use of POUserFunc. And you need to take special care about Algebraic funcs and
their return types. The algebraic funcs have a return type but they are of the type java.lang.reflect.Type.
I guess you need to figure out a way to convert between Type and pig types. Alan can you comment
on this?

3) Also I was wondering as to why we need the getReturnType, finish, outputSchema & isAsynchronous
methods? I thought POUserFunc would not expose these methods as a wrapper object because POUserFunc
complies with our operator model and the operator model doesn't support these methods. Alan,
comments needed. Can you think of some scenario where these would be needed in our current
operator model?

> Rework physical plan
> --------------------
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch, incr4.patch,
incr5.patch, MRCompilerTests_PlansAndOutputs.txt, Phy_AbsClass.patch, physicalOps.patch, physicalOps.patch,
podistinct.patch, pogenerate.patch, pogenerate.patch, pogenerate.patch, posort.patch
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message