impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4586: don't constant fold in backend
Date Wed, 07 Dec 2016 02:12:48 GMT
Tim Armstrong has uploaded a new change for review.

Change subject: IMPALA-4586: don't constant fold in backend

IMPALA-4586: don't constant fold in backend

Before this patch, the backend would cache values based on IsConstant().
This meant that there was no way to override caching of values of
non-deterministic UDFs, e.g. with enable_expr_rewrites.

After this patch, we only cache literal values in the backend. This
offers the same performance as before, since the frontend should
constant fold all constant expressions anyway.

Also rename some functions to more cleanly separate the backend concepts
of "constant" expressions and expressions that can be evaluated without
a TupleRow. In a future change we should remove the IsConstant()
analysis logic from the backend entirely and pass the information from
the frontend.

Added targeted test to test constant folding of UDFs: we expect
different results depending on whether constant folding is enabled.

Also run TestUdfs with expr rewrites enabled and disabled, since this
can exercise different code paths. Refactored test_udfs somewhat to
avoid running uninteresting combinations of query options for
targeted tests and removed some 'drop * if not exists' statements
that aren't necessary when using unique_database.

This change revealed flakiness in test_mem_limit, which seems
to have only worked by coincidence. Updated TrackAllocation() to
actually set the query status when a memory limit is exceeded.
Looped this test for a while to make sure it isn't flaky any

Also fix other test bugs where the vector argument is modified
in-place, which can leak out to other tests.

Change-Id: I0c76e3c8a8d92749256c312080ecd7aac5d99ce7
M be/src/exprs/
M be/src/exprs/expr-context.h
M be/src/exprs/
M be/src/exprs/expr.h
M be/src/exprs/
M be/src/exprs/literal.h
M be/src/exprs/
M be/src/exprs/null-literal.h
M be/src/exprs/
M be/src/exprs/
M be/src/service/
M be/src/testutil/
M be/src/udf/udf-internal.h
M be/src/udf/
M common/thrift/ImpalaInternalService.thrift
M fe/src/main/java/org/apache/impala/analysis/
M fe/src/main/java/org/apache/impala/analysis/
M fe/src/main/java/org/apache/impala/analysis/
M fe/src/main/java/org/apache/impala/analysis/
M fe/src/main/java/org/apache/impala/service/
A testdata/workloads/functional-query/queries/QueryTest/udf-init-close-deterministic.test
M testdata/workloads/functional-query/queries/QueryTest/udf-init-close.test
A testdata/workloads/functional-query/queries/QueryTest/udf-non-deterministic.test
M testdata/workloads/functional-query/queries/QueryTest/udf.test
M tests/common/
M tests/query_test/
26 files changed, 463 insertions(+), 403 deletions(-)

  git pull ssh:// refs/changes/91/5391/3
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c76e3c8a8d92749256c312080ecd7aac5d99ce7
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <>

View raw message