impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marcel Kornacker (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4270: Gracefully fail unsupported queries with mt dop > 0.
Date Wed, 12 Oct 2016 04:33:45 GMT
Marcel Kornacker has posted comments on this change.

Change subject: IMPALA-4270: Gracefully fail unsupported queries with mt_dop > 0.
......................................................................


Patch Set 2:

(10 comments)

http://gerrit.cloudera.org:8080/#/c/4677/2/be/src/exec/exec-node.cc
File be/src/exec/exec-node.cc:

Line 272:           DCHECK_EQ(state->query_options().num_scanner_threads, 1);
roll into a single dcheck with a compound predicate


http://gerrit.cloudera.org:8080/#/c/4677/2/common/thrift/PlanNodes.thrift
File common/thrift/PlanNodes.thrift:

Line 205:   
trailing


Line 207:   // If this is true then the MT_DOP query option must be > 0.
leave todo that it should be removed later


http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
File fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java:

Line 314:    * Retrurns a set of file formats being scanned.
spelling

the set


Line 562:     msg.hdfs_scan_node.setUse_mt_scan_nodeIsSet(useMtScanNode_);
..IsSet? it doesn't look like you're calling the actual setter


http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/Planner.java
File fe/src/main/java/org/apache/impala/planner/Planner.java:

Line 206:     if (ctx_.getQueryOptions().mt_dop > 0) {
the if is redundant, but you can make it a checkstate at the start of the function, if you
want.

also, the planner only creates parallel plans for query stmts.


http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/PlannerContext.java
File fe/src/main/java/org/apache/impala/planner/PlannerContext.java:

Line 83:   public TQueryOptions getQueryOptions() { return getRootAnalyzer().getQueryOptions();
}
why the change?


http://gerrit.cloudera.org:8080/#/c/4677/2/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java:

Line 155:    * - MT_DOP > 0 is not supported for plans with distributed joins or table
sinks.
it's only supported for QueryStmts. take a look at Frontend.createExecRequest()


http://gerrit.cloudera.org:8080/#/c/4677/1/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test:

Line 5: ---- PLAN
> The purpose is to validate that this plan is executable with mt_dop>0 and n
but it's not executable, because the join sink isn't executable.

sorry, i overlooked that earlier: for the upcoming release we need to turn this off even for
non-distributed joins.


http://gerrit.cloudera.org:8080/#/c/4677/2/testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test
File testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test:

Line 51: |  tuple-ids=0,1 row-size=8B cardinality=unavailable
what happened here?


-- 
To view, visit http://gerrit.cloudera.org:8080/4677
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I91a60ea7b6e3ae4ee44be856615ddd3cd0af476d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.behm@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message