hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-161) Rework physical plan
Date Wed, 16 Apr 2008 01:26:23 GMT

    [ https://issues.apache.org/jira/browse/PIG-161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12589382#action_12589382
] 

Alan Gates commented on PIG-161:
--------------------------------

On the relational operator name, that's a standard database term for operators that operate
on relations (ie tables), so this would include select/filter, join, aggregate, sort, and
project.  When I use the term I'm stretching it a bit, applying it to operators that operate
on bags (which are equivalent to tables).  I don't have any particular need to call them relational
operators, and if people who don't come from a database background (which will probably be
most pig users and developers) find that name confusing I'm fine with using a different name.
 I agree that some refactoring will be necessary and that we can do that in one fell swoop
towards the end.

Shubham and Shravan had asked me to think about whether the interface to EvalFunc should change
as part of the rewrite of the pipeline.  I think the interface will need to change in that
operations will need to be supported for the type checker, so that the typechecker can query
the object and determine what arguments it expects so that it can see if it's being passed
the right thing.  But I don't think the exec() method itself (which I assume is what you're
interested in) needs to change.  Basically this method has to allow some way to pass a variable
number of arguments to the UDF and trust that the UDF either knows what's coming or knows
how to handle whatever it may get.  Passing a tuple as an argument is as good a way as any
to do this.  It's heavier weight than passing an array (which is what I'd probably do if I
was starting from scratch), but not heavier enough weight to justify changing the interface.
  Creating a new tuple does not require copying the data, just adding a set of pointers to
it.

> Rework physical plan
> --------------------
>
>                 Key: PIG-161
>                 URL: https://issues.apache.org/jira/browse/PIG-161
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: arithmeticOperators.patch, incr2.patch, incr3.patch, incr4.patch,
Phy_AbsClass.patch, pogenerate.patch, pogenerate.patch, pogenerate.patch
>
>
> This bug tracks work to rework all of the physical operators as described in http://wiki.apache.org/pig/PigTypesFunctionalSpec

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message