impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4080, IMPALA-3638: Introduce ExecNode::Codegen()
Date Thu, 13 Oct 2016 16:19:25 GMT
Michael Ho has uploaded a new patch set (#4).

Change subject: IMPALA-4080, IMPALA-3638: Introduce ExecNode::Codegen()

IMPALA-4080, IMPALA-3638: Introduce ExecNode::Codegen()

This patch is mostly mechanical move of codegen related logic
from each exec node's Prepare() to its Codegen() function.
After this change, code generation will no longer happen in
Prepare(). Instead, it will happen after Prepare() completes in
PlanFragmentExecutor. This is an intermediate step towards the
final goal of sharing compiled code among fragment instances in

As part of the clean up, this change also removes the logic for
lazy codegen object creation. In other words, if codegen is enabled,
the codegen object will always be created. This simplifies some
of the logic in ScalarFnCall::Prepare() and various Codegen()
functions by reducing error checking needed. This change also
removes the logic added for tackling IMPALA-1755 as it's not
needed anymore after the clean up.

The clean up also rectifies a not so well documented situation.
Previously, even if a user explicitly sets DISABLE_CODEGEN to true,
we may still codegen a UDF if it was written in LLVM IR or if it
has more than 8 arguments. This patch enforces the query option
by failing the query in both cases. To run the query, the user
must enable codegen. This change also extends the number of
arguments supported in the interpretation path of ScalarFn to 20.

Change-Id: I207566bc9f4c6a159271ecdbc4bbdba3d78c6651
M be/src/exec/
M be/src/exec/aggregation-node.h
M be/src/exec/
M be/src/exec/exchange-node.h
M be/src/exec/
M be/src/exec/exec-node.h
M be/src/exec/
M be/src/exec/hash-join-node.h
M be/src/exec/
M be/src/exec/hash-table.h
M be/src/exec/
M be/src/exec/hdfs-avro-scanner.h
M be/src/exec/
M be/src/exec/
M be/src/exec/hdfs-scan-node-base.h
M be/src/exec/
M be/src/exec/
M be/src/exec/
M be/src/exec/
M be/src/exec/old-hash-table.h
M be/src/exec/
M be/src/exec/partitioned-aggregation-node.h
M be/src/exec/
M be/src/exec/partitioned-hash-join-builder.h
M be/src/exec/
M be/src/exec/partitioned-hash-join-node.h
M be/src/exec/
M be/src/exec/sort-node.h
M be/src/exec/
M be/src/exec/topn-node.h
M be/src/exprs/
M be/src/exprs/case-expr.h
M be/src/exprs/
M be/src/exprs/compound-predicates.h
M be/src/exprs/
M be/src/exprs/conditional-functions.h
M be/src/exprs/
M be/src/exprs/expr.h
M be/src/exprs/
M be/src/exprs/hive-udf-call.h
M be/src/exprs/
M be/src/exprs/is-not-empty-predicate.h
M be/src/exprs/
M be/src/exprs/literal.h
M be/src/exprs/
M be/src/exprs/null-literal.h
M be/src/exprs/
M be/src/exprs/scalar-fn-call.h
M be/src/exprs/
M be/src/exprs/slot-ref.h
M be/src/exprs/
M be/src/exprs/tuple-is-null-predicate.h
M be/src/runtime/
M be/src/runtime/
M be/src/runtime/runtime-state.h
M be/src/runtime/sorted-run-merger.h
M be/src/runtime/
M be/src/runtime/sorter.h
M be/src/runtime/
M be/src/runtime/tuple.h
M be/src/service/
M be/src/testutil/
M be/src/util/
M be/src/util/tuple-row-compare.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/
M testdata/workloads/functional-query/queries/QueryTest/udf-errors.test
M testdata/workloads/functional-query/queries/QueryTest/udf.test
M tests/query_test/
69 files changed, 639 insertions(+), 686 deletions(-)

  git pull ssh:// refs/changes/51/4651/4
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I207566bc9f4c6a159271ecdbc4bbdba3d78c6651
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Michael Ho <>
Gerrit-Reviewer: Michael Ho <>
Gerrit-Reviewer: Tim Armstrong <>

View raw message