hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shravan Matthur Narayanamurthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-161) Rework physical plan
Date Wed, 23 Apr 2008 13:09:21 GMT

    [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12591635#action_12591635
] 

Shravan Matthur Narayanamurthy commented on PIG-161:
----------------------------------------------------

Patch contaning POSort, PODistinct and POUserFunc with their respective test cases.

Comments:
1) @Author tags need to be removed.
2) TODO tags need to be removed. I am guessing its just an artifact of using eclipse. Are
there things you need to do at all the places where the TODO tag appears?
3) I wasn't sure if the POSort and POUserFunc were serializable. This is essential as the
plan will reconstructed in the map/reduce phases by serialization/deserialization.
4) POUserFunc.getNext(Long) uses super.getNext(). I guess it should have been just getNext()
instead.
5) A suggestion on the Algebraic eval funcs. The current POUserFunc just supports the retrieval
of the appropriate objects. I was hoping for it to encapsulate this in the getNext(). Something
like this:
Have a variable denote whether the initial, intmd or final function is needed from the POUserFunc
instance. In the set method of this variable, initialize your func object by doing the relevant
get method(getInitial, getIntmd or getFinal). Other than that the rest of the code should
work encapsulating the Algebraic functionality.
6) In UDFSortComparator, I think if there is an exception, the code will break with a null
pointer exception as you will be trying to access res.result and res is null.
7) I also saw printStackTrace somewhere. i guess it should replace with appropriate log/throw
8) In POSort, if it has to be used in MapReduce, I think the UserComparator has to be accessible.
I don't see a way of accessing it.

> Rework physical plan
> --------------------
>
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch, incr4.patch,
incr5.patch, MRCompilerTests_PlansAndOutputs.txt, Phy_AbsClass.patch, physicalOps.patch, podistinct.patch,
pogenerate.patch, pogenerate.patch, pogenerate.patch, posort.patch
>
>
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message