hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-157) Add types and rework execution pipeline
Date Tue, 20 May 2008 23:28:55 GMT

    [ https://issues.apache.org/jira/browse/PIG-157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12598526#action_12598526
] 

Alan Gates commented on PIG-157:
--------------------------------

With my latest checkin (revision 658485) we are getting close to tying all the pieces together.
 I know of the following outstanding issues:

# HExecutionEngine and LocalExecutionEngine compile() methods need to plug into Shubham's
logical to physical translator.  Shubham has submitted a patch with the translator but was
still seeing some unit test failures, so I have not tied it in yet.
# explain needs to be fleshed out in HExecutionEngine and LocalExecutionEngine.  To do this,
POPrinter will have to be converted to the new physical operators and MRPrinter will have
to be written (it's just a shell right now).
# FileLocalizer.openDFSFile() needs access to JobConf in order to open the file.  Previously
it went through PigRecordReader, but this was a hack.  Shravan is looking at how this should
be done.
# Currently logical plans returned by registerQuery aren't being stitched together.  Santhosh
is looking at how to resolve this.  Once that is resolved the resulting plan need to be passed
to the validators and then on to the appropriate execution engine.
# PigServer.totalHadoopTimeSpent doesn't work yet.  Shravan changed Launcher so that it tracks
time spent, but PigServer does not have a reference to the launcher there.  Need to figure
out how to get it a reference to the launcher object, or maybe that value should be static,
not sure.
# 2 final tests in TestBuiltin are failing.  They are commented out for now.  But it looks
to me like they should pass.  They fail in the parser.  Santhosh, could you take a look at
this.



> Add types and rework execution pipeline
> ---------------------------------------
>
>                 Key: PIG-157
>                 URL: https://issues.apache.org/jira/browse/PIG-157
>             Project: Pig
>          Issue Type: New Feature
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: Core.patch.zip, exceptions.patch, incr1.zip
>
>
> This is the tracking bug for the work to add types to pig and rework the execution pipeline.
 Individual components of this work are covered in subtasks.
> Functional and design specs for this work are:
> http://wiki.apache.org/pig/PigTypesFunctionalSpec
> http://wiki.apache.org/pig/PigTypesDesign
> http://wiki.apache.org/pig/PigExecutionModel
> This work is being done on the branch types, since it is large and disruptive, and we
want to be able to do incremental checkins without causing issues for the trunk.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message