hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Ryaboy <dvrya...@gmail.com>
Subject Re: Bug in new logical optimizer framework?
Date Fri, 02 Jul 2010 00:58:44 GMT
Renato,
I just want to make sure folks know -- Pig already has a number of such
optimizations. Daniel's work is aimed at making it (much) easier to write
such rules and to add a couple new ones. But some of the classic
optimizations like projection and filter push-down already exist in the
released versions of Pig.

D

On Thu, Jul 1, 2010 at 5:32 PM, Daniel Dai <jianyong@yahoo-inc.com> wrote:

> Yes, they are classic logical optimization plus some Pig only optimization.
> All these are rule based. https://issues.apache.org/jira/browse/PIG-1319is an umbrella
Jira to track all new optimization rules.
>
> Daniel
>
>
> Renato MarroquĂ­n Mogrovejo wrote:
>
>> Hi,
>>
>> I am also interested in this logical plan optimization framework
>> functionality. You mentioned that rules are being developed, could you
>> explain a little bit more about them? Are they like the classic logical
>> optimizations (early projection, early filtering, among others)?
>> Thanks in advance.
>>
>> Renato M.
>>
>> logical plan optimization framework
>>
>> 2010/6/28 Alan Gates <gates@yahoo-inc.com>
>>
>>
>>
>>> On Jun 28, 2010, at 12:36 AM, Swati Jain wrote:
>>>
>>>  Thanks for the prompt reply. As you mentioned optimization is in its
>>>
>>>
>>>> developing stage, does it mean optimization framework is not complete or
>>>> only rules are in developing stage? In addition to that, I would really
>>>> appreciate if you could give a rough idea when the patch will be
>>>> available
>>>> and what functionality will it contain?
>>>>
>>>>  At this point we believe the framework is complete and rules are being
>>>>
>>>>
>>> developed.  But the framework has never been used in user testing
>>> situations
>>> (alpha or beta testing) so there will be a whole round of bugs to fix
>>> once
>>> that testing is done.
>>>
>>> The current plan is to switch to this code as the actual optimizer with
>>> 0.8, which we hope to release late this year (no promises).
>>>
>>> Alan.
>>>
>>>
>>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message