impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2502: don't redundantly repartition grouping aggregations
Date Thu, 10 Mar 2016 00:41:07 GMT
Hello Alex Behm,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2414

to look at the new patch set (#8).

Change subject: IMPALA-2502: don't redundantly repartition grouping aggregations
......................................................................

IMPALA-2502: don't redundantly repartition grouping aggregations

Grouping aggregations previously always repartitioned their input,
even if preceding joins or aggs had already partitioned the data on the
required key (or an equivalent key). This patch checks to see if data is
already partitioned on the required exprs (or equivalent ones), and if
so skips the preaggregation and only does a merge aggregation.

The patch also does some refactoring of the aggregation planning in
DistributedPlanner to make it easier to implement the change.

Includes planner tests for the three cases that are affected:
grouping aggregations, non-grouping distinct aggregations and
grouping distinct aggregations.

Change-Id: Iffdcfd3629b8a69bd23915e1adba3b8323cbbaef
---
M fe/src/main/java/com/cloudera/impala/analysis/AggregateInfo.java
M fe/src/main/java/com/cloudera/impala/planner/AnalyticPlanner.java
M fe/src/main/java/com/cloudera/impala/planner/DataPartition.java
M fe/src/main/java/com/cloudera/impala/planner/DistributedPlanner.java
M testdata/workloads/functional-planner/queries/PlannerTest/aggregation.test
M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
7 files changed, 286 insertions(+), 105 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/14/2414/8
-- 
To view, visit http://gerrit.cloudera.org:8080/2414
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iffdcfd3629b8a69bd23915e1adba3b8323cbbaef
Gerrit-PatchSet: 8
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message