impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dimitris Tsirogiannis (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-3548: Prune runtime filters based on query options in the FE
Date Tue, 29 Aug 2017 15:16:49 GMT
Dimitris Tsirogiannis has posted comments on this change.

Change subject: IMPALA-3548: Prune runtime filters based on query options in the FE
......................................................................


Patch Set 2:

(4 comments)

Just some answers on the comments. Haven't looked at the new patch yet.

http://gerrit.cloudera.org:8080/#/c/7564/2/fe/src/main/java/org/apache/impala/planner/Planner.java
File fe/src/main/java/org/apache/impala/planner/Planner.java:

PS2, Line 119: singleNodePlan
> The single node plan is created on L104 and assigned to 'singleNodePlan'. D
Let's expand to the comment to mention that 'singleNodePlan' now points to the root of the
distributed plan.


PS2, Line 117: // create runtime filters
             :     if (ctx_.getQueryOptions().getRuntime_filter_mode() != TRuntimeFilterMode.OFF)
{
             :       RuntimeFilterGenerator.generateRuntimeFilters(ctx_, singleNodePlan);
             :       ctx_.getAnalysisResult().getTimeline().markEvent("Runtime filters computed");
             :     }
> The block was moved here because the distributed plan is created on L114 an
Thanks for explaining.


http://gerrit.cloudera.org:8080/#/c/7564/2/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java
File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java:

PS2, Line 165: tFilterTarget.setIs_bound_by_partition_columns(isBoundByPartitionColumns);
             :         tFilterTarget.setIs_local_target(isLocalTarget);
> The BE uses these flags to create instances of Coordinator::FilterTarget th
I see. Thanks


http://gerrit.cloudera.org:8080/#/c/7564/2/tests/query_test/test_runtime_filters.py
File tests/query_test/test_runtime_filters.py:

PS2, Line 87: def test_distrib_plan_filter_pruning(self, vector):
            :     vector.get_value('exec_option')['num_nodes'] = 0
            :     self.run_test_case('QueryTest/runtime_filters_distrib_pruning', vector)
            : 
            :   def test_singlenode_plan_filter_pruning(self, vector):
            :     vector.get_value('exec_option')['num_nodes'] = 1
            :     self.run_test_case('QueryTest/runtime_filters_singlenode_pruning', vector)
> These are written as query tests and not planner tests because the planner 
I don't follow. Did you check how the query option is set in testRuntimeFilterPropagation()
in PlannerTest.java?


-- 
To view, visit http://gerrit.cloudera.org:8080/7564
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id0f0b200e02442edcad8df3979f652d66c6e52eb
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Attila Jeges <attilaj@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Attila Jeges <attilaj@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message