impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Skye Wanderman-Milne (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2784: codegen PartitionedHashJoinNode::Partition::BuildHashTable()
Date Mon, 25 Apr 2016 22:05:43 GMT
Hello Jim Apple, Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/2113

to look at the new patch set (#6).

Change subject: IMPALA-2784: codegen PartitionedHashJoinNode::Partition::BuildHashTable()
......................................................................

IMPALA-2784: codegen PartitionedHashJoinNode::Partition::BuildHashTable()

This patch implements codegen for (part of) the BuildHashTable()
function by codegening a version of the new InsertBatch() function,
which is called by BuildHashTableInternal(). The codegen'd
InsertBatch() is based on the cross-compiled original, and replaces
EvalBuildRow(), Equals(), and HashCurrentRow() calls.

For the following benchmark query:
 SELECT STRAIGHT_JOIN count(*)
 FROM tpch_parquet.orders l1
 JOIN [broadcast] biglineitem ON l_orderkey = o_orderkey
 WHERE l_partkey > 0
 AND l_suppkey > 0
 AND l_linenumber > 0
 AND o_orderdate < '19000'
 AND l_orderkey < 350000000;

Hash table build time went from 7.4s to 3.7s (2x faster), and overall
query time went from 18.1s to 14.1s (28% faster). This increased
codegen time from 212ms to 279ms (30% slower).

This patch also changes the BUILD_RUNTIME_FILTERS conditional to be
based on an input argument rather than a template parameter. This
slightly slows down execution but as the benefit of requiring fewer
codegen'd functions.

Change-Id: I616f46a861b4909d7a6e66dcf947b3518556768e
---
M be/src/codegen/gen_ir_descriptions.py
M be/src/exec/partitioned-hash-join-node-ir.cc
M be/src/exec/partitioned-hash-join-node.cc
M be/src/exec/partitioned-hash-join-node.h
4 files changed, 121 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/13/2113/6
-- 
To view, visit http://gerrit.cloudera.org:8080/2113
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I616f46a861b4909d7a6e66dcf947b3518556768e
Gerrit-PatchSet: 6
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbapple@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mmokhtar@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message