impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4731: Crash when sorting on non-deterministic expr
Date Mon, 06 Feb 2017 22:02:06 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-4731: Crash when sorting on non-deterministic expr
......................................................................


Patch Set 1:

(4 comments)

http://gerrit.cloudera.org:8080/#/c/5914/1//COMMIT_MSG
Commit Message:

Line 7: IMPALA-4731: Crash when sorting on non-deterministic expr
This also may fix IMPALA-4728 or IMPALA-397.

Might be good to add a targeted-perf test to confirm it fixes the perf bug IMPALA-4728. I
think Greg Rahn had some examples of analytic functions where lazy materialisation was very
slow.


http://gerrit.cloudera.org:8080/#/c/5914/1/fe/src/main/java/org/apache/impala/analysis/SortInfo.java
File fe/src/main/java/org/apache/impala/analysis/SortInfo.java:

Line 194:       } else {
This is a weird corner case case, but I think we should drop LiteralExprs instead of materialising
them. It's plausible that real-world queries might have order-by expressions that can be rewritten
into constants.


Line 207:     substituteOrderingExprs(substOrderBy, analyzer);
I think we only need to substitute the SlotRefs in the first branch of the if() above. We
could just do the substitution there. Otherwise it's a bit hard to reason about materializedOrderingExprs
containing a mix of SlotRefs referring to the old and new tuples.


http://gerrit.cloudera.org:8080/#/c/5914/1/tests/query_test/test_sort.py
File tests/query_test/test_sort.py:

Line 162:   def test_order_by_random(self):
I think the example in IMPALA-397 would also be a good test - there rand() is in the select
list so we can verify that the order is correct.


-- 
To view, visit http://gerrit.cloudera.org:8080/5914
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5dcda32fc7770d42fc500ce87fc54d58e5b5dc00
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message