Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201703020211.v222BkYF007536@ip-10-146-233-104.ec2.internal>
Date: Thu, 2 Mar 2017 02:11:46 +0000
From: "Tim Armstrong (Code Review)" <gerrit@cloudera.org>
To: Joe McDonnell <joemcdonnell@cloudera.com>, impala-cr@cloudera.com,
        reviews@impala.incubator.apache.org
CC: Alex Behm <alex.behm@cloudera.com>,
        Mostafa Mokhtar <mmokhtar@cloudera.com>,
        Matthew Mulder <mmulder@cloudera.com>,
        Marcel Kornacker <marcel@cloudera.com>, Lars Volker <lv@cloudera.com>
Reply-To: tarmstrong@cloudera.com
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4624=3A_Implement_Parquet_dictionary_filtering=0A?=
In-Reply-To: <gerrit.1486172553000.I3a7cc3bd0523fbf3c79bd924219e909ef671cfd7@gerrit.cloudera.org>
References: <gerrit.1486172553000.I3a7cc3bd0523fbf3c79bd924219e909ef671cfd7@gerrit.cloudera.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.12.7
archived-at: Thu, 02 Mar 2017 02:11:52 -0000

Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4624: Implement Parquet dictionary filtering
......................................................................


Patch Set 14:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/5904/14/fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java
File fe/src/main/java/org/apache/impala/analysis/FunctionCallExpr.java:

Line 243:   public boolean isDeterministic() {
> Switched this over to isBuiltinRandom/isBuiltinRandomFn. I agree that it is
We ran into a similar problem with non-deterministic UDFs when we added expression rewrites. We never provided any guarantees about how non-deterministic UDFs were evaluated, and it was already inconsistent. E.g. partitions have been statically pruned based on UDF output since many releases ago, so we don't guarantee that predicates with a non-deterministic UDF are evaluated against every row anyway.

The solution we went before was to change the behaviour (since it wasn't contractual) but provide the enable_expr_rewrites safety valve in case someone was relying on the behaviour.

I believe we have a JIRA somewhere to add metadata about whether a UDF is non-determistic.


-- 
To view, visit http://gerrit.cloudera.org:8080/5904
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I3a7cc3bd0523fbf3c79bd924219e909ef671cfd7
Gerrit-PatchSet: 14
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell <joemcdonnell@cloudera.com>
Gerrit-Reviewer: Lars Volker <lv@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Matthew Mulder <mmulder@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mmokhtar@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes