hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alan Gates (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-262) Pig needs an optimizer
Date Tue, 01 Jul 2008 15:20:44 GMT

    [ https://issues.apache.org/jira/browse/PIG-262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12609619#action_12609619

Alan Gates commented on PIG-262:

I think I agree, though I don't want to start mixing up those frameworks right at the moment.
 I want to focus on getting what we have working first.  They are slightly different, one
focuses on checking the tree while one focuses on rearranging it.  But both end up adding
nodes to the plan.  And thus they will both need tools to patch up graph connections and schemas
after these changes.  It's not clear whether we should maintain the distinction and have a
framework that can run both or whether we should merge the two into one class.  I think the
former, but I'm not sure.

> Pig needs an optimizer
> ----------------------
>                 Key: PIG-262
>                 URL: https://issues.apache.org/jira/browse/PIG-262
>             Project: Pig
>          Issue Type: Sub-task
>          Components: impl
>            Reporter: Alan Gates
>            Assignee: Alan Gates
>         Attachments: optimizer.patch
> We need to add an optimizer to pig.  This will enable us to do some traditional optimizations,
such as filter and projection pushing, join order and execution choices, etc.  It will also
enable optimizations specific to map/reduce (such as using the combiner).
> The optimizer will need to operate at various levels, including the logical, physical,
and possibly map/reduce plan levels.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message