Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <201710262054.v9QKsxT0009206@ip-10-146-233-104.ec2.internal>
Date: Thu, 26 Oct 2017 20:54:59 +0000
From: "Thomas Tauber-Marshall (Code Review)" <gerrit@cloudera.org>
To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org
CC: Matthew Jacobs <mjacobs@apache.org>, Michael Ho <kwho@cloudera.com>,
        Mostafa Mokhtar <mmokhtar@cloudera.com>, Lars Volker <lv@cloudera.com>,
        Tim Armstrong <tarmstrong@cloudera.com>, Todd Lipcon <todd@apache.org>
Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4252=3A_Min-max_runtime_filters_for_Kudu=0A?=
In-Reply-To: <gerrit.1503529605000.I02bad890f5b5f78388a3041bf38f89369b5e2f1c@gerrit.cloudera.org>
References: <gerrit.1503529605000.I02bad890f5b5f78388a3041bf38f89369b5e2f1c@gerrit.cloudera.org>
Reply-To: tmarshall@cloudera.com, impala-cr@cloudera.com, lv@cloudera.com,
        marcelk@gmail.com, kwho@cloudera.com, tarmstrong@cloudera.com,
        mmokhtar@cloudera.com, todd@apache.org,
        reviews@impala.incubator.apache.org, mjacobs@apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
User-Agent: Gerrit/2.14.2
Content-Type: multipart/alternative; boundary="ai+j/l78DME="; charset=UTF-8
archived-at: Thu, 26 Oct 2017 20:55:05 -0000

--ai+j/l78DME=
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thomas Tauber-Marshall has posted comments on this change=2E ( http://gerri=
t=2Ecloudera=2Eorg:8080/7793 )

Change subject: IMPALA-4252: Min-max runtim=
e filters for Kudu
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=
=2E=2E


Patch Set 8:

> > Patch Set 7:
 > >
 > > > > Patch Set 7:
 > >  > =
>
 > >  > > Perf results:
 > >  > > =2E=2E=2E
 > >  >
 > >  > I'm surprised=
 that only a few queries saw significant
 > speedups=2E Is
 > >  > this in =
line with what you saw with Parquet runtime filters on
 > >  > TPC-H? Or ar=
e we losing a lot by using min/max instead of
 > bloom or
 > >  > in-list s=
tyle filters?
 > >
 > > Not sure about bloom filters perf, though I can run=
 those numbers
 > for comparison=2E
 > 
 > I haven't looked at this patch, =
but had a question about the
 > design:
 > 
 > Are we still pushing blooms =
across a join to prevent shuffling of
 > data? Or are we now pushing _only_=
 min/max?
 > 
 > It seems there is value in pushing both: the bloom for eva=
luation
 > on the other side of the join to prevent shuffling, and the min/=
max
 > to push all the way to the scanner to reduce I/O=2E
 > 
 > Not sure =
if the patch is already doing this=2E

Impala only evaluates runtime filter=
s in the scan=2E Even prior to this patch, the Kudu scanner was not evaluat=
ing bloom filters (and hash joins with Kudu scan targets don't build bloom =
filters)=2E

It certainly could be useful to evaluate bloom filters on the =
Impala side of a Kudu scan, but I believe our thinking was that it wasn't w=
orth it to implement that - better to just wait until bloom filters can be =
pushed all the way down into Kudu=2E If bloom filters in Kudu are a long wa=
y off, though, we should maybe reevaluate that=2E


-- 
To view, visit http=
://gerrit=2Ecloudera=2Eorg:8080/7793
To unsubscribe, visit http://gerrit=2E=
cloudera=2Eorg:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: mas=
ter
Gerrit-MessageType: comment
Gerrit-Change-Id: I02bad890f5b5f78388a3041b=
f38f89369b5e2f1c
Gerrit-Change-Number: 7793
Gerrit-PatchSet: 8
Gerrit-Owner=
: Thomas Tauber-Marshall <tmarshall@cloudera=2Ecom>
Gerrit-Reviewer: Anonym=
ous Coward #345
Gerrit-Reviewer: Lars Volker <lv@cloudera=2Ecom>
Gerrit-Rev=
iewer: Matthew Jacobs <mjacobs@apache=2Eorg>
Gerrit-Reviewer: Michael Ho <k=
who@cloudera=2Ecom>
Gerrit-Reviewer: Mostafa Mokhtar <mmokhtar@cloudera=2Ec=
om>
Gerrit-Reviewer: Thomas Tauber-Marshall <tmarshall@cloudera=2Ecom>
Gerr=
it-Reviewer: Tim Armstrong <tarmstrong@cloudera=2Ecom>
Gerrit-Reviewer: Tod=
d Lipcon <todd@apache=2Eorg>
Gerrit-Comment-Date: Thu, 26 Oct 2017 20:54:59=
 +0000
Gerrit-HasComments: No

--ai+j/l78DME=--