hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashutosh Chauhan (JIRA)" <>
Subject [jira] [Commented] (HIVE-2340) optimize orderby followed by a groupby
Date Thu, 17 Jan 2013 22:14:12 GMT


Ashutosh Chauhan commented on HIVE-2340:

Thanks Yin for explaining. Your ASCII art helped in understanding the differences : ) I better
understand the reason for the fake new operator now. I think in cases you have pointed out
when there is such kind of trees, this reduce deduplication approach won't help, since it
looks at linear chain of RS and eliminates the one where it could. You would need a fake operator
in such case because you don't want to modify the GBY or Join operators which make sense.
I see the merits of Ysmart better now.

Though, on the other hand patch on this jira is still useful and complementary to ysmart.
Since, it will collapse linear RS, instead of adding fake ones. In addition to collapsing
of those operators, it will also make the life of ysmart easier because than ysmart will be
dealing with simpler plans with reduce sinks already deduplicated. We need to make sure reducededup
rule fires before ysmart for both optimizations to play nicely. So, I think we should make
progress on both these patches.

[~navis] Will you like to refresh this patch?
> optimize orderby followed by a groupby
> --------------------------------------
>                 Key: HIVE-2340
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Query Processor
>            Reporter: Navis
>            Assignee: Navis
>            Priority: Minor
>              Labels: perfomance
>         Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2340.D1209.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2340.D1209.2.patch,
ASF.LICENSE.NOT.GRANTED--HIVE-2340.D1209.5.patch, HIVE-2340.1.patch.txt
> Before implementing optimizer for JOIN-GBY, try to implement RS-GBY optimizer(cluster-by
following group-by).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message