impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joe McDonnell (Code Review)" <>
Subject [Impala-ASF-CR] IMPALA-4996: Single-threaded KuduScanNode
Date Wed, 15 Mar 2017 16:31:58 GMT
Hello Marcel Kornacker,

I'd like you to reexamine a change.  Please visit

to look at the new patch set (#6).

Change subject: IMPALA-4996: Single-threaded KuduScanNode

IMPALA-4996: Single-threaded KuduScanNode

This introduces KuduScanNodeMt, the single-threaded version
of KuduScanNode that materializes the tuples in GetNext().
KuduScanNodeMt is enabled by the same condition as
HdfsScanNodeMt: mt_dop is greater than or equal to 1.

To share code between the two implementations, KuduScanNode
and KuduScanNodeMt are now subclasses of KuduScanNodeBase,
which implements the shared code. The KuduScanner is
minimally impacted, as it already had the required GetNext

Since the KuduClient is a heavy-weight object, it is now
shared at the QueryState level. We try to share the
KuduClient as much as possible, but there are times when
the KuduClient cannot be shared. Each Kudu table has
master addresses stored in the Hive Metastore. We only
share KuduClients for tables that have an identical value
for the master addresses. In the ideal case, every Kudu
table will have the same value, but there is no explicit
guarantee of this.

The testing for this is a modified version of
kudu-scan-node.test run with various mt_dop values.

Change-Id: I6e4593300e376bc508b78acaea64ffdd2c73a67a
M be/src/exec/CMakeLists.txt
M be/src/exec/
M be/src/exec/hdfs-scan-node-base.h
A be/src/exec/
A be/src/exec/kudu-scan-node-base.h
A be/src/exec/
A be/src/exec/kudu-scan-node-mt.h
M be/src/exec/
M be/src/exec/kudu-scan-node.h
M be/src/exec/
M be/src/exec/kudu-scanner.h
M be/src/runtime/
M be/src/runtime/query-state.h
M common/thrift/PlanNodes.thrift
M fe/src/main/java/org/apache/impala/planner/
A testdata/workloads/functional-query/queries/QueryTest/mt-dop-kudu.test
M tests/query_test/
17 files changed, 558 insertions(+), 143 deletions(-)

  git pull ssh:// refs/changes/12/6312/6
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I6e4593300e376bc508b78acaea64ffdd2c73a67a
Gerrit-PatchSet: 6
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Joe McDonnell <>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell <>
Gerrit-Reviewer: Marcel Kornacker <>
Gerrit-Reviewer: Matthew Jacobs <>
Gerrit-Reviewer: Thomas Tauber-Marshall <>

View raw message