hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato Marroquín Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Bug in new logical optimizer framework?
Date Sun, 04 Jul 2010 06:35:55 GMT
Thanks Daniel and Dmitriy for your answers, now I have a much clear idea of
what type of optimization work is being done on PIG. And just another quick
question, do you guys know where in the code those optimizations are?
I just want to give it a closer look (:

Renato M.

2010/7/1 Dmitriy Ryaboy <dvryaboy@gmail.com>

> Renato,
> I just want to make sure folks know -- Pig already has a number of such
> optimizations. Daniel's work is aimed at making it (much) easier to write
> such rules and to add a couple new ones. But some of the classic
> optimizations like projection and filter push-down already exist in the
> released versions of Pig.
>
> D
>
> On Thu, Jul 1, 2010 at 5:32 PM, Daniel Dai <jianyong@yahoo-inc.com> wrote:
>
> > Yes, they are classic logical optimization plus some Pig only
> optimization.
> > All these are rule based.
> https://issues.apache.org/jira/browse/PIG-1319is an umbrella Jira to track
> all new optimization rules.
> >
> > Daniel
> >
> >
> > Renato Marroquín Mogrovejo wrote:
> >
> >> Hi,
> >>
> >> I am also interested in this logical plan optimization framework
> >> functionality. You mentioned that rules are being developed, could you
> >> explain a little bit more about them? Are they like the classic logical
> >> optimizations (early projection, early filtering, among others)?
> >> Thanks in advance.
> >>
> >> Renato M.
> >>
> >> logical plan optimization framework
> >>
> >> 2010/6/28 Alan Gates <gates@yahoo-inc.com>
> >>
> >>
> >>
> >>> On Jun 28, 2010, at 12:36 AM, Swati Jain wrote:
> >>>
> >>>  Thanks for the prompt reply. As you mentioned optimization is in its
> >>>
> >>>
> >>>> developing stage, does it mean optimization framework is not complete
> or
> >>>> only rules are in developing stage? In addition to that, I would
> really
> >>>> appreciate if you could give a rough idea when the patch will be
> >>>> available
> >>>> and what functionality will it contain?
> >>>>
> >>>>  At this point we believe the framework is complete and rules are
> being
> >>>>
> >>>>
> >>> developed.  But the framework has never been used in user testing
> >>> situations
> >>> (alpha or beta testing) so there will be a whole round of bugs to fix
> >>> once
> >>> that testing is done.
> >>>
> >>> The current plan is to switch to this code as the actual optimizer with
> >>> 0.8, which we hope to release late this year (no promises).
> >>>
> >>> Alan.
> >>>
> >>>
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message