impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5483: Automatically disable codegen for small queries
Date Thu, 22 Jun 2017 00:02:34 GMT
Tim Armstrong has posted comments on this change.

Change subject: IMPALA-5483: Automatically disable codegen for small queries
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/7153/3//COMMIT_MSG
Commit Message:

PS3, Line 15: a plan node
> Multi-column PK is not uncommon. Even with accurate stats, Planner could ge
I hear your concerns and I do think we should think it through carefully. I also think we
shouldn't be too scared to change defaults when the current defaults don't make really sense.
I'd struggle to explain to someone why Impala spends multiple seconds doing codegen when the
planner estimates show that it's a small query that will take 10s or 100s of milliseconds
to execute. It's easy to explain that Impala decided to disable codegen because the estimates
showed it to be the right decision (but the estimates were wrong).

If it's a bad fit for a particular workload it's possible to disable this or change the threshold.
The current threshold is pretty safe according to my experiments.

The motivation is that now if you have a workload with a mix of big and small queries, you
get stuck in a situation where the small queries are burning lots of CPU doing unnecessary
codegen. The only workaround is to manually disable codegen for the small queries, but that's
not really practical to do in most cases.

RE: disabling it for joins. Part of the motivation was that the small query optimisation doesn't
work for joins. It was disabled because of a bug with num_nodes=1.

RE: explain plans. It does show up in profiles with a string like "codegen disabled due to
optimization hints" for the plan nodes. I could add it to explain plans too but it wasn't
clear if it was worth adding even more detail to them.


-- 
To view, visit http://gerrit.cloudera.org:8080/7153
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I273bcee58641f5b97de52c0b2caab043c914b32e
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Juan Yu <jyu@cloudera.com>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message