impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Tauber-Marshall (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2805: Order filters based on selectivity and cost
Date Tue, 22 Mar 2016 00:02:07 GMT
Thomas Tauber-Marshall has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/2598

Change subject: IMPALA-2805: Order filters based on selectivity and cost
......................................................................

IMPALA-2805: Order filters based on selectivity and cost

Added costs to all Exprs, which estimate the relative cost of evaluating
an expression and all of its subexpressions. Costs are calculated when the
computeCost function is called. For now, these costs are intended as a simple
way to order expressions from cheap to expensive, not necessarily to be a
precise reflection of running times.

In general, expressions that deal with complex types like strings and
timestamps will have higher cost than those dealing with simple types
like numbers and booleans. Additionally, expressions with complicated
subexpressions will have higher cost than simpler expressions.

Also added PlanNode.orderConjuncts, which takes a list of Exprs and returns
a new list sorted according to an estimate of the cheapest order to evaulate
the conjuncts in, based on their cost and selectivity.

Exprs with selectivity estimates will always appear first in the sorted order.
These are sorted by repeatedly iterating over them and choosing the conjunct
that would result in the least total estimated work were it to be applied before
the remaining conjuncts. Exprs without selectivity estimates will appear at the
end of the sorted order, in order of cost.

Change-Id: I02279a26fbc6308ac5eb819d78345fc010469034
---
M fe/src/main/java/com/cloudera/impala/analysis/AnalyticExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/ArithmeticExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/BetweenPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/BinaryPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/BoolLiteral.java
M fe/src/main/java/com/cloudera/impala/analysis/CaseExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/CastExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/CompoundPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/ExistsPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/Expr.java
M fe/src/main/java/com/cloudera/impala/analysis/FunctionCallExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/InPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/IsNotEmptyPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/IsNullPredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/LikePredicate.java
M fe/src/main/java/com/cloudera/impala/analysis/NullLiteral.java
M fe/src/main/java/com/cloudera/impala/analysis/NumericLiteral.java
M fe/src/main/java/com/cloudera/impala/analysis/SlotRef.java
M fe/src/main/java/com/cloudera/impala/analysis/StringLiteral.java
M fe/src/main/java/com/cloudera/impala/analysis/Subquery.java
M fe/src/main/java/com/cloudera/impala/analysis/TimestampArithmeticExpr.java
M fe/src/main/java/com/cloudera/impala/analysis/TupleIsNullPredicate.java
M fe/src/main/java/com/cloudera/impala/planner/AggregationNode.java
M fe/src/main/java/com/cloudera/impala/planner/AnalyticEvalNode.java
M fe/src/main/java/com/cloudera/impala/planner/DataSourceScanNode.java
M fe/src/main/java/com/cloudera/impala/planner/EmptySetNode.java
M fe/src/main/java/com/cloudera/impala/planner/HBaseScanNode.java
M fe/src/main/java/com/cloudera/impala/planner/HashJoinNode.java
M fe/src/main/java/com/cloudera/impala/planner/HdfsScanNode.java
M fe/src/main/java/com/cloudera/impala/planner/NestedLoopJoinNode.java
M fe/src/main/java/com/cloudera/impala/planner/PlanNode.java
M fe/src/main/java/com/cloudera/impala/planner/SelectNode.java
M fe/src/main/java/com/cloudera/impala/planner/SortNode.java
M fe/src/main/java/com/cloudera/impala/planner/SubplanNode.java
M fe/src/main/java/com/cloudera/impala/planner/UnionNode.java
A fe/src/test/java/com/cloudera/impala/planner/OrderConjunctsTest.java
M testdata/workloads/functional-planner/queries/PlannerTest/analytic-fns.test
M testdata/workloads/functional-planner/queries/PlannerTest/data-source-tables.test
M testdata/workloads/functional-planner/queries/PlannerTest/distinct.test
M testdata/workloads/functional-planner/queries/PlannerTest/hbase.test
M testdata/workloads/functional-planner/queries/PlannerTest/inline-view.test
M testdata/workloads/functional-planner/queries/PlannerTest/join-order.test
M testdata/workloads/functional-planner/queries/PlannerTest/joins.test
M testdata/workloads/functional-planner/queries/PlannerTest/nested-collections.test
M testdata/workloads/functional-planner/queries/PlannerTest/nested-loop-join.test
M testdata/workloads/functional-planner/queries/PlannerTest/outer-joins.test
M testdata/workloads/functional-planner/queries/PlannerTest/predicate-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/runtime-filter-propagation.test
M testdata/workloads/functional-planner/queries/PlannerTest/subquery-rewrite.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpcds-all.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-all.test
M testdata/workloads/functional-planner/queries/PlannerTest/tpch-nested.test
M testdata/workloads/functional-planner/queries/PlannerTest/union.test
53 files changed, 554 insertions(+), 208 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/98/2598/1
-- 
To view, visit http://gerrit.cloudera.org:8080/2598
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I02279a26fbc6308ac5eb819d78345fc010469034
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Thomas Tauber-Marshall <tmarshall@cloudera.com>

Mime
View raw message