impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Skye Wanderman-Milne (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-2548: Codegen Tuple::MaterializeExprs() and use in TopN node
Date Thu, 05 May 2016 22:02:15 GMT
Hello Marcel Kornacker, Michael Ho, Internal Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/1901

to look at the new patch set (#17).

Change subject: IMPALA-2548: Codegen Tuple::MaterializeExprs() and use in TopN node
......................................................................

IMPALA-2548: Codegen Tuple::MaterializeExprs() and use in TopN node

For the following benchmark query:
 select count(*) from (select l_orderkey from biglineitem order by l_orderkey limit 1000)
a

The overall query time goes from 2.74s to 1.74s, with the top-n node
time going from 2.2s to 1.0s. There is no effect on sort node time.

The overall approach of this patch is to move the
TopNNode::InsertTupleRow() call into a cross-compiled batched function
(InsertBatch()), and then replace the MaterializeExprs() calls with
new functions built using the IRBuilder. This involves new codegen
utilities, such as CodegenAnyVal::WriteToSlot() and the ability to
hardcode in a MemPool pointer from which to make varlen data
allocations. This patch also adds a new timer measuring the time spent
inserting tuple rows.

The existing TestQueries::test_top_n and TestQueries::test_sort tests
pass with this patch.

Change-Id: Ib422a8d50303c21c6a228675157bf867e8619444
---
M be/src/codegen/codegen-anyval.cc
M be/src/codegen/codegen-anyval.h
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/exec/CMakeLists.txt
A be/src/exec/topn-node-ir.cc
M be/src/exec/topn-node.cc
M be/src/exec/topn-node.h
M be/src/runtime/descriptors.cc
M be/src/runtime/descriptors.h
M be/src/runtime/mem-pool.h
M be/src/runtime/raw-value.cc
M be/src/runtime/sorter.cc
M be/src/runtime/tuple.cc
M be/src/runtime/tuple.h
17 files changed, 514 insertions(+), 96 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/01/1901/17
-- 
To view, visit http://gerrit.cloudera.org:8080/1901
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib422a8d50303c21c6a228675157bf867e8619444
Gerrit-PatchSet: 17
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Skye Wanderman-Milne <skye@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Marcel Kornacker <marcel@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <skye@cloudera.com>

Mime
View raw message