Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 626A8200B81 for ; Tue, 13 Sep 2016 22:59:24 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 5DF84160AD2; Tue, 13 Sep 2016 20:59:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 58BED160AAA for ; Tue, 13 Sep 2016 22:59:23 +0200 (CEST) Received: (qmail 37300 invoked by uid 500); 13 Sep 2016 20:59:22 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 37283 invoked by uid 99); 13 Sep 2016 20:59:22 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 13 Sep 2016 20:59:22 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id C915BC03BC for ; Tue, 13 Sep 2016 20:59:21 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.162 X-Spam-Level: * X-Spam-Status: No, score=1.162 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx2-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id QJLlaPeX3fdT for ; Tue, 13 Sep 2016 20:59:19 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx2-lw-us.apache.org (ASF Mail Server at mx2-lw-us.apache.org) with ESMTPS id 6AEA55F64D for ; Tue, 13 Sep 2016 20:59:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id u8DKxIrA026113; Tue, 13 Sep 2016 20:59:18 GMT Message-Id: <201609132059.u8DKxIrA026113@ip-10-146-233-104.ec2.internal> Date: Tue, 13 Sep 2016 20:59:18 +0000 From: "Tim Armstrong (Code Review)" To: Michael Ho , impala-cr@cloudera.com, reviews@impala.incubator.apache.org CC: Dan Hecht Reply-To: tarmstrong@cloudera.com X-Gerrit-MessageType: newpatchset Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-4008=3A_don=27t_bake_in_hash_table_and_hash_join_pointers=0A?= X-Gerrit-Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5 X-Gerrit-ChangeURL: X-Gerrit-Commit: 13f84ef7f42dfcb1c3d074deb212a8879cdb1469 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.2 archived-at: Tue, 13 Sep 2016 20:59:24 -0000 Hello Michael Ho, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/4326 to look at the new patch set (#5). Change subject: IMPALA-4008: don't bake in hash table and hash join pointers ...................................................................... IMPALA-4008: don't bake in hash table and hash join pointers This fixes some of the cases where memory addresses are baked into codegen'd code. Testing: Ran exhaustive build. Perf: Ran a local perf run. No significant changes. I was able to see some small improvements on microbenchmarks. +-----------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +-----------+-----------------------+---------+------------+------------+----------------+ | TPCH(_20) | parquet / none / none | 9.07 | +0.46% | 5.88 | +0.34% | +-----------+-----------------------+---------+------------+------------+----------------+ +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ | Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters | +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ | TPCH(_20) | TPCH-Q2 | parquet / none / none | 2.12 | 1.89 | +12.29% | * 10.85% * | * 20.30% * | 1 | 10 | | TPCH(_20) | TPCH-Q13 | parquet / none / none | 9.84 | 9.34 | +5.39% | 9.01% | 3.79% | 1 | 10 | | TPCH(_20) | TPCH-Q17 | parquet / none / none | 14.61 | 14.19 | +2.97% | 2.15% | 1.52% | 1 | 10 | | TPCH(_20) | TPCH-Q18 | parquet / none / none | 14.76 | 14.35 | +2.82% | 3.20% | 2.64% | 1 | 10 | | TPCH(_20) | TPCH-Q9 | parquet / none / none | 13.72 | 13.54 | +1.30% | 1.75% | 0.70% | 1 | 10 | | TPCH(_20) | TPCH-Q8 | parquet / none / none | 5.71 | 5.64 | +1.30% | 1.21% | 1.23% | 1 | 10 | | TPCH(_20) | TPCH-Q19 | parquet / none / none | 47.35 | 46.75 | +1.28% | 2.39% | 1.88% | 1 | 10 | | TPCH(_20) | TPCH-Q5 | parquet / none / none | 4.57 | 4.52 | +1.20% | 1.30% | 0.88% | 1 | 10 | | TPCH(_20) | TPCH-Q16 | parquet / none / none | 2.07 | 2.05 | +1.12% | 2.59% | 1.79% | 1 | 10 | | TPCH(_20) | TPCH-Q11 | parquet / none / none | 1.45 | 1.45 | +0.15% | 2.69% | 2.06% | 1 | 10 | | TPCH(_20) | TPCH-Q3 | parquet / none / none | 4.65 | 4.65 | -0.09% | 2.12% | 2.17% | 1 | 10 | | TPCH(_20) | TPCH-Q4 | parquet / none / none | 3.22 | 3.23 | -0.26% | 1.03% | 1.33% | 1 | 10 | | TPCH(_20) | TPCH-Q7 | parquet / none / none | 15.84 | 15.92 | -0.50% | 0.91% | 1.15% | 1 | 10 | | TPCH(_20) | TPCH-Q14 | parquet / none / none | 3.29 | 3.31 | -0.59% | 3.31% | 1.58% | 1 | 10 | | TPCH(_20) | TPCH-Q22 | parquet / none / none | 2.65 | 2.67 | -0.78% | 3.03% | 1.46% | 1 | 10 | | TPCH(_20) | TPCH-Q15 | parquet / none / none | 4.50 | 4.55 | -1.19% | 2.87% | 2.45% | 1 | 10 | | TPCH(_20) | TPCH-Q20 | parquet / none / none | 3.84 | 3.91 | -1.76% | 2.20% | 1.94% | 1 | 10 | | TPCH(_20) | TPCH-Q10 | parquet / none / none | 5.58 | 5.70 | -2.00% | 1.01% | 1.79% | 1 | 10 | | TPCH(_20) | TPCH-Q21 | parquet / none / none | 22.84 | 23.42 | -2.47% | 0.68% | 0.56% | 1 | 10 | | TPCH(_20) | TPCH-Q1 | parquet / none / none | 11.25 | 11.60 | -3.06% | 0.48% | 1.74% | 1 | 10 | | TPCH(_20) | TPCH-Q12 | parquet / none / none | 3.81 | 3.98 | -4.38% | 1.62% | 1.14% | 1 | 10 | | TPCH(_20) | TPCH-Q6 | parquet / none / none | 1.94 | 2.04 | -4.85% | 2.40% | 1.58% | 1 | 10 | +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ +--------------------+-----------------------+---------+------------+------------+----------------+ | Workload | File Format | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) | +--------------------+-----------------------+---------+------------+------------+----------------+ | TARGETED-PERF(_20) | parquet / none / none | 8.17 | -1.66% | 2.96 | -1.48% | +--------------------+-----------------------+---------+------------+------------+----------------+ +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ | Workload | Query | File Format | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients | Iters | +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ | TARGETED-PERF(_20) | primitive_topn_bigint | parquet / none / none | 3.32 | 2.87 | +15.31% | * 17.22% * | 1.64% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q4 | parquet / none / none | 7.07 | 6.61 | +6.93% | * 15.59% * | 5.08% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q1 | parquet / none / none | 1.18 | 1.12 | +5.57% | 1.94% | 2.96% | 1 | 10 | | TARGETED-PERF(_20) | primitive_empty_build_join_1 | parquet / none / none | 10.75 | 10.47 | +2.76% | 1.15% | 0.94% | 1 | 10 | | TARGETED-PERF(_20) | primitive_top-n_all | parquet / none / none | 24.30 | 23.85 | +1.87% | 1.40% | 0.82% | 1 | 10 | | TARGETED-PERF(_20) | primitive_broadcast_join_2 | parquet / none / none | 2.45 | 2.42 | +1.38% | 1.93% | 1.33% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_bigint_selective | parquet / none / none | 0.57 | 0.57 | +1.16% | 3.66% | 4.19% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q3 | parquet / none / none | 1.70 | 1.68 | +1.06% | 1.88% | 2.69% | 1 | 10 | | TARGETED-PERF(_20) | primitive_broadcast_join_3 | parquet / none / none | 4.15 | 4.13 | +0.47% | 1.27% | 1.28% | 1 | 10 | | TARGETED-PERF(_20) | primitive_broadcast_join_1 | parquet / none / none | 1.46 | 1.46 | +0.32% | 1.68% | 2.43% | 1 | 10 | | TARGETED-PERF(_20) | primitive_orderby_all | parquet / none / none | 12.92 | 12.89 | +0.22% | 1.69% | 1.12% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q4 | parquet / none / none | 1.68 | 1.67 | +0.21% | 2.31% | 2.44% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q7 | parquet / none / none | 3.35 | 3.35 | +0.14% | 1.10% | 1.84% | 1 | 10 | | TARGETED-PERF(_20) | primitive_conjunct_ordering_4 | parquet / none / none | 0.46 | 0.46 | +0.07% | 0.29% | 0.27% | 1 | 10 | | TARGETED-PERF(_20) | primitive_groupby_bigint_pk | parquet / none / none | 35.99 | 36.04 | -0.13% | 8.55% | 6.95% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q6 | parquet / none / none | 0.97 | 0.97 | -0.16% | 2.68% | 2.66% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_decimal_selective | parquet / none / none | 0.84 | 0.84 | -0.31% | 2.85% | 3.58% | 1 | 10 | | TARGETED-PERF(_20) | primitive_shuffle_join_union_all_with_groupby | parquet / none / none | 20.46 | 20.53 | -0.34% | 0.60% | 0.60% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q7 | parquet / none / none | 0.98 | 0.98 | -0.44% | 2.78% | 2.33% | 1 | 10 | | TARGETED-PERF(_20) | primitive_exchange_shuffle | parquet / none / none | 25.21 | 25.34 | -0.49% | 1.94% | 1.30% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_string_like | parquet / none / none | 6.27 | 6.31 | -0.56% | 0.59% | 0.43% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q5 | parquet / none / none | 1.96 | 1.97 | -0.60% | 1.98% | 1.25% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_bigint_non_selective | parquet / none / none | 0.53 | 0.53 | -0.71% | 2.64% | 0.28% | 1 | 10 | | TARGETED-PERF(_20) | primitive_exchange_broadcast | parquet / none / none | 14.77 | 14.97 | -1.33% | 2.59% | 2.39% | 1 | 10 | | TARGETED-PERF(_20) | PERF_LIMIT-Q1 | parquet / none / none | 0.01 | 0.01 | -1.40% | 3.77% | 3.51% | 1 | 10 | | TARGETED-PERF(_20) | primitive_orderby_bigint | parquet / none / none | 2.58 | 2.61 | -1.44% | 2.85% | 1.04% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q1 | parquet / none / none | 1.47 | 1.49 | -1.63% | 2.79% | 1.62% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_decimal_non_selective | parquet / none / none | 0.79 | 0.81 | -1.65% | 1.91% | 2.67% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q2 | parquet / none / none | 1.56 | 1.58 | -1.73% | 2.86% | 2.57% | 1 | 10 | | TARGETED-PERF(_20) | PERF_STRING-Q6 | parquet / none / none | 4.48 | 4.59 | -2.22% | 1.21% | 0.85% | 1 | 10 | | TARGETED-PERF(_20) | primitive_conjunct_ordering_5 | parquet / none / none | 11.80 | 12.10 | -2.50% | 3.06% | 2.09% | 1 | 10 | | TARGETED-PERF(_20) | primitive_conjunct_ordering_1 | parquet / none / none | 7.96 | 8.18 | -2.58% | 1.69% | 2.17% | 1 | 10 | | TARGETED-PERF(_20) | primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none / none | 93.11 | 95.78 | -2.78% | 1.07% | 1.00% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_string_selective | parquet / none / none | 0.88 | 0.91 | -3.03% | 4.68% | 5.34% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q5 | parquet / none / none | 2.44 | 2.52 | -3.35% | 2.39% | 2.04% | 1 | 10 | | TARGETED-PERF(_20) | primitive_groupby_bigint_highndv | parquet / none / none | 9.16 | 9.50 | -3.59% | 0.98% | 1.56% | 1 | 10 | | TARGETED-PERF(_20) | primitive_conjunct_ordering_3 | parquet / none / none | 1.44 | 1.50 | -4.14% | 1.10% | 1.08% | 1 | 10 | | TARGETED-PERF(_20) | primitive_filter_string_non_selective | parquet / none / none | 0.89 | 0.93 | -4.31% | 7.20% | 5.12% | 1 | 10 | | TARGETED-PERF(_20) | primitive_groupby_decimal_highndv | parquet / none / none | 13.98 | 14.94 | -6.40% | * 11.24% * | 8.66% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q2 | parquet / none / none | 2.87 | 3.13 | -8.36% | 0.57% | 1.59% | 1 | 10 | | TARGETED-PERF(_20) | primitive_groupby_bigint_lowndv | parquet / none / none | 1.48 | 1.62 | -8.91% | 1.62% | 1.28% | 1 | 10 | | TARGETED-PERF(_20) | primitive_conjunct_ordering_2 | parquet / none / none | 13.68 | 15.05 | -9.14% | 2.91% | 1.83% | 1 | 10 | | TARGETED-PERF(_20) | primitive_groupby_decimal_lowndv.test | parquet / none / none | 1.47 | 1.61 | -9.26% | 0.19% | 1.50% | 1 | 10 | | TARGETED-PERF(_20) | PERF_AGG-Q3 | parquet / none / none | 4.25 | 4.82 | -11.82% | 0.79% | 0.84% | 1 | 10 | +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+ Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5 --- M be/src/exec/hash-table-test.cc M be/src/exec/hash-table.cc M be/src/exec/hash-table.h M be/src/exec/hash-table.inline.h M be/src/exec/partitioned-aggregation-node-ir.cc M be/src/exec/partitioned-aggregation-node.cc M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc 8 files changed, 270 insertions(+), 279 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/4326/5 -- To view, visit http://gerrit.cloudera.org:8080/4326 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5 Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong