impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4624: Implement Parquet dictionary filtering
Date Thu, 02 Mar 2017 02:11:46 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4624: Implement Parquet dictionary filtering

Patch Set 14:

(1 comment)
File fe/src/main/java/org/apache/impala/analysis/

Line 243:   public boolean isDeterministic() {
> Switched this over to isBuiltinRandom/isBuiltinRandomFn. I agree that it is
We ran into a similar problem with non-deterministic UDFs when we added expression rewrites.
We never provided any guarantees about how non-deterministic UDFs were evaluated, and it was
already inconsistent. E.g. partitions have been statically pruned based on UDF output since
many releases ago, so we don't guarantee that predicates with a non-deterministic UDF are
evaluated against every row anyway.

The solution we went before was to change the behaviour (since it wasn't contractual) but
provide the enable_expr_rewrites safety valve in case someone was relying on the behaviour.

I believe we have a JIRA somewhere to add metadata about whether a UDF is non-determistic.

To view, visit
To unsubscribe, visit

Gerrit-MessageType: comment
Gerrit-Change-Id: I3a7cc3bd0523fbf3c79bd924219e909ef671cfd7
Gerrit-PatchSet: 14
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <>
Gerrit-Reviewer: Alex Behm <>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Lars Volker <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Matthew Mulder <>
Gerrit-Reviewer: Mostafa Mokhtar <>
Gerrit-Reviewer: Tim Armstrong <>
Gerrit-HasComments: Yes

View raw message