hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-1178) LogicalPlan and Optimizer are too complex and hard to work with
Date Wed, 03 Feb 2010 02:06:19 GMT

    [ https://issues.apache.org/jira/browse/PIG-1178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12828895#action_12828895
] 

Ashutosh Chauhan commented on PIG-1178:
---------------------------------------

Last patch was more then 5500 lines. Phew! To make it easier to read, I have few minor comments
which you probably can sneak in your next upload:
a) There is commented code at various places. Deleting those lines will make patch smaller.
b) At various places Runtime Exceptions are thrown. Do we need a new Exception class something
like ExperimentalOptimizerException so as to easily spot those exceptions? That will aid in
debugging. Also error messages are pretty terse. More details can be put in messages, again
to aid in debugging later on.
c) At couple of places, changes are made outside of experimental package. Will be useful to
put comments there for why are those needed. 

> LogicalPlan and Optimizer are too complex and hard to work with
> ---------------------------------------------------------------
>
>                 Key: PIG-1178
>                 URL: https://issues.apache.org/jira/browse/PIG-1178
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Alan Gates
>            Assignee: Ying He
>         Attachments: expressions-2.patch, expressions.patch, lp.patch, lp.patch, pig_1178.patch,
PIG_1178.patch
>
>
> The current implementation of the logical plan and the logical optimizer in Pig has proven
to not be easily extensible. Developer feedback has indicated that adding new rules to the
optimizer is quite burdensome. In addition, the logical plan has been an area of numerous
bugs, many of which have been difficult to fix. Developers also feel that the logical plan
is difficult to understand and maintain. The root cause for these issues is that a number
of design decisions that were made as part of the 0.2 rewrite of the front end have now proven
to be sub-optimal. The heart of this proposal is to revisit a number of those proposals and
rebuild the logical plan with a simpler design that will make it much easier to maintain the
logical plan as well as extend the logical optimizer. 
> See http://wiki.apache.org/pig/PigLogicalPlanOptimizerRewrite for full details.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message