impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Tauber-Marshall (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3629: Codegen TransferScratchTuples() in hdfs-parquet-scanner
Date Mon, 22 Aug 2016 21:54:45 GMT
Hello Michael Ho, Matthew Jacobs, Tim Armstrong,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/3774

to look at the new patch set (#10).

Change subject: IMPALA-3629: Codegen TransferScratchTuples() in hdfs-parquet-scanner
......................................................................

IMPALA-3629: Codegen TransferScratchTuples() in hdfs-parquet-scanner

We currently don't do any codegen in the hdfs-parquet scanner,
which limits performance. This patch creates a new function,
ProcessScratchBatch, which contains the inner loop of
TransferScratchTuples, allowing us to cross-compile only this
performance critical part. It also uses CodegenEvalConjuncts to
replace the call to EvalConjuncts in ProcessScratchBatch with a
codegen'd version.

Additionally, it modifies the Codegen functions in
hdfs-avro/text/sequence-scanner to take an output parameter for
the codegen'd function and return a Status in order to improve
logging around codegen failures.

Change-Id: Ic327e437c7cd2b3f92cdb11c1e907bfee2d44ee8
---
M be/src/codegen/gen_ir_descriptions.py
M be/src/codegen/impala-ir.cc
M be/src/exec/CMakeLists.txt
M be/src/exec/hdfs-avro-scanner.cc
M be/src/exec/hdfs-avro-scanner.h
A be/src/exec/hdfs-parquet-scanner-ir.cc
M be/src/exec/hdfs-parquet-scanner.cc
M be/src/exec/hdfs-parquet-scanner.h
M be/src/exec/hdfs-scan-node.cc
M be/src/exec/hdfs-scanner.cc
M be/src/exec/hdfs-scanner.h
M be/src/exec/hdfs-sequence-scanner.cc
M be/src/exec/hdfs-sequence-scanner.h
M be/src/exec/hdfs-text-scanner.cc
M be/src/exec/hdfs-text-scanner.h
15 files changed, 277 insertions(+), 128 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/74/3774/10
-- 
To view, visit http://gerrit.cloudera.org:8080/3774
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic327e437c7cd2b3f92cdb11c1e907bfee2d44ee8
Gerrit-PatchSet: 10
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <mj@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarshall@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message