impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Henry Robinson (Code Review)" <>
Subject [Impala-CR](cdh5-trunk) IMPALA-3077: Enable runtime filters when PHJ spills
Date Fri, 15 Apr 2016 18:16:14 GMT
Hello Tim Armstrong,

I'd like you to reexamine a change.  Please visit

to look at the new patch set (#4).

Change subject: IMPALA-3077: Enable runtime filters when PHJ spills

IMPALA-3077: Enable runtime filters when PHJ spills

This patch changes when runtime filters are produced in the partitioned
hash-join node to allow filters to be produced even when the PHJ
spills. Filters are now produced during the level0 processing of the
PHJ's build-side input in ProcessBuildBatchWithFilters().

Since this function is codegen'ed, so now is filter production. We use
constant-propagation via constant argument injection to disable filter
production at no cost when it is not needed (including in level1+
repartitioning). I inspected the IR to confirm that the constant
propagation works as expected.

This change also allows us to send filters earlier during build-side

This patch also fixes a bug where re-partitioning beyond level0 would
not use the codegen'ed implementation of ProcessBuildBatch().

A new test is added to test_runtime_row_filters, for Parquet only, which
spills and confirms that filtering still occurs.

Change-Id: I59a2d9ee03ccea6b674392584e4c7f272233571e
M be/src/exec/
M be/src/exec/
M be/src/exec/partitioned-hash-join-node.h
M testdata/workloads/functional-query/queries/QueryTest/runtime_row_filters.test
4 files changed, 87 insertions(+), 81 deletions(-)

  git pull ssh:// refs/changes/83/2783/4
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I59a2d9ee03ccea6b674392584e4c7f272233571e
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Henry Robinson <>
Gerrit-Reviewer: Dan Hecht <>
Gerrit-Reviewer: Henry Robinson <>
Gerrit-Reviewer: Tim Armstrong <>

View raw message