impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-4008: don't bake in hash table and hash join pointers
Date Tue, 13 Sep 2016 20:59:18 GMT
Hello Michael Ho,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/4326

to look at the new patch set (#5).

Change subject: IMPALA-4008: don't bake in hash table and hash join pointers
......................................................................

IMPALA-4008: don't bake in hash table and hash join pointers

This fixes some of the cases where memory addresses are baked into
codegen'd code.

Testing:
Ran exhaustive build.

Perf:
Ran a local perf run. No significant changes. I was able to see some
small improvements on microbenchmarks.

    +-----------+-----------------------+---------+------------+------------+----------------+
    | Workload  | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean)
|
    +-----------+-----------------------+---------+------------+------------+----------------+
    | TPCH(_20) | parquet / none / none | 9.07    | +0.46%     | 5.88       | +0.34%     
   |
    +-----------+-----------------------+---------+------------+------------+----------------+

    +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | Workload  | Query    | File Format           | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)
 | Base StdDev(%) | Num Clients | Iters |
    +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | TPCH(_20) | TPCH-Q2  | parquet / none / none | 2.12   | 1.89        |   +12.29%  | *
10.85% * | * 20.30% *     | 1           | 10    |
    | TPCH(_20) | TPCH-Q13 | parquet / none / none | 9.84   | 9.34        |   +5.39%   | 
 9.01%    |   3.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q17 | parquet / none / none | 14.61  | 14.19       |   +2.97%   | 
 2.15%    |   1.52%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q18 | parquet / none / none | 14.76  | 14.35       |   +2.82%   | 
 3.20%    |   2.64%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q9  | parquet / none / none | 13.72  | 13.54       |   +1.30%   | 
 1.75%    |   0.70%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q8  | parquet / none / none | 5.71   | 5.64        |   +1.30%   | 
 1.21%    |   1.23%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q19 | parquet / none / none | 47.35  | 46.75       |   +1.28%   | 
 2.39%    |   1.88%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q5  | parquet / none / none | 4.57   | 4.52        |   +1.20%   | 
 1.30%    |   0.88%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q16 | parquet / none / none | 2.07   | 2.05        |   +1.12%   | 
 2.59%    |   1.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q11 | parquet / none / none | 1.45   | 1.45        |   +0.15%   | 
 2.69%    |   2.06%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q3  | parquet / none / none | 4.65   | 4.65        |   -0.09%   | 
 2.12%    |   2.17%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q4  | parquet / none / none | 3.22   | 3.23        |   -0.26%   | 
 1.03%    |   1.33%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q7  | parquet / none / none | 15.84  | 15.92       |   -0.50%   | 
 0.91%    |   1.15%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q14 | parquet / none / none | 3.29   | 3.31        |   -0.59%   | 
 3.31%    |   1.58%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q22 | parquet / none / none | 2.65   | 2.67        |   -0.78%   | 
 3.03%    |   1.46%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q15 | parquet / none / none | 4.50   | 4.55        |   -1.19%   | 
 2.87%    |   2.45%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q20 | parquet / none / none | 3.84   | 3.91        |   -1.76%   | 
 2.20%    |   1.94%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q10 | parquet / none / none | 5.58   | 5.70        |   -2.00%   | 
 1.01%    |   1.79%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q21 | parquet / none / none | 22.84  | 23.42       |   -2.47%   | 
 0.68%    |   0.56%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q1  | parquet / none / none | 11.25  | 11.60       |   -3.06%   | 
 0.48%    |   1.74%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q12 | parquet / none / none | 3.81   | 3.98        |   -4.38%   | 
 1.62%    |   1.14%        | 1           | 10    |
    | TPCH(_20) | TPCH-Q6  | parquet / none / none | 1.94   | 2.04        |   -4.85%   | 
 2.40%    |   1.58%        | 1           | 10    |
    +-----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

    +--------------------+-----------------------+---------+------------+------------+----------------+
    | Workload           | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean)
|
    +--------------------+-----------------------+---------+------------+------------+----------------+
    | TARGETED-PERF(_20) | parquet / none / none | 8.17    | -1.66%     | 2.96       | -1.48%
        |
    +--------------------+-----------------------+---------+------------+------------+----------------+

    +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | Workload           | Query                                                  | File Format
          | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)  | Base StdDev(%) | Num Clients
| Iters |
    +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
    | TARGETED-PERF(_20) | primitive_topn_bigint                                  | parquet
/ none / none | 3.32   | 2.87        |   +15.31%  | * 17.22% * |   1.64%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q4                                            | parquet
/ none / none | 7.07   | 6.61        |   +6.93%   | * 15.59% * |   5.08%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q1                                            | parquet
/ none / none | 1.18   | 1.12        |   +5.57%   |   1.94%    |   2.96%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_empty_build_join_1                           | parquet
/ none / none | 10.75  | 10.47       |   +2.76%   |   1.15%    |   0.94%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_top-n_all                                    | parquet
/ none / none | 24.30  | 23.85       |   +1.87%   |   1.40%    |   0.82%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_2                             | parquet
/ none / none | 2.45   | 2.42        |   +1.38%   |   1.93%    |   1.33%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_bigint_selective                      | parquet
/ none / none | 0.57   | 0.57        |   +1.16%   |   3.66%    |   4.19%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q3                                         | parquet
/ none / none | 1.70   | 1.68        |   +1.06%   |   1.88%    |   2.69%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_3                             | parquet
/ none / none | 4.15   | 4.13        |   +0.47%   |   1.27%    |   1.28%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_broadcast_join_1                             | parquet
/ none / none | 1.46   | 1.46        |   +0.32%   |   1.68%    |   2.43%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_orderby_all                                  | parquet
/ none / none | 12.92  | 12.89       |   +0.22%   |   1.69%    |   1.12%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q4                                         | parquet
/ none / none | 1.68   | 1.67        |   +0.21%   |   2.31%    |   2.44%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q7                                         | parquet
/ none / none | 3.35   | 3.35        |   +0.14%   |   1.10%    |   1.84%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_4                          | parquet
/ none / none | 0.46   | 0.46        |   +0.07%   |   0.29%    |   0.27%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_pk                            | parquet
/ none / none | 35.99  | 36.04       |   -0.13%   |   8.55%    |   6.95%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q6                                            | parquet
/ none / none | 0.97   | 0.97        |   -0.16%   |   2.68%    |   2.66%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_decimal_selective                     | parquet
/ none / none | 0.84   | 0.84        |   -0.31%   |   2.85%    |   3.58%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_shuffle_join_union_all_with_groupby          | parquet
/ none / none | 20.46  | 20.53       |   -0.34%   |   0.60%    |   0.60%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q7                                            | parquet
/ none / none | 0.98   | 0.98        |   -0.44%   |   2.78%    |   2.33%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_exchange_shuffle                             | parquet
/ none / none | 25.21  | 25.34       |   -0.49%   |   1.94%    |   1.30%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_like                           | parquet
/ none / none | 6.27   | 6.31        |   -0.56%   |   0.59%    |   0.43%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q5                                         | parquet
/ none / none | 1.96   | 1.97        |   -0.60%   |   1.98%    |   1.25%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_bigint_non_selective                  | parquet
/ none / none | 0.53   | 0.53        |   -0.71%   |   2.64%    |   0.28%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_exchange_broadcast                           | parquet
/ none / none | 14.77  | 14.97       |   -1.33%   |   2.59%    |   2.39%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_LIMIT-Q1                                          | parquet
/ none / none | 0.01   | 0.01        |   -1.40%   |   3.77%    |   3.51%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_orderby_bigint                               | parquet
/ none / none | 2.58   | 2.61        |   -1.44%   |   2.85%    |   1.04%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q1                                         | parquet
/ none / none | 1.47   | 1.49        |   -1.63%   |   2.79%    |   1.62%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_decimal_non_selective                 | parquet
/ none / none | 0.79   | 0.81        |   -1.65%   |   1.91%    |   2.67%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q2                                         | parquet
/ none / none | 1.56   | 1.58        |   -1.73%   |   2.86%    |   2.57%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_STRING-Q6                                         | parquet
/ none / none | 4.48   | 4.59        |   -2.22%   |   1.21%    |   0.85%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_5                          | parquet
/ none / none | 11.80  | 12.10       |   -2.50%   |   3.06%    |   2.09%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_1                          | parquet
/ none / none | 7.96   | 8.18        |   -2.58%   |   1.69%    |   2.17%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_shuffle_join_one_to_many_string_with_groupby | parquet
/ none / none | 93.11  | 95.78       |   -2.78%   |   1.07%    |   1.00%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_selective                      | parquet
/ none / none | 0.88   | 0.91        |   -3.03%   |   4.68%    |   5.34%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q5                                            | parquet
/ none / none | 2.44   | 2.52        |   -3.35%   |   2.39%    |   2.04%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_highndv                       | parquet
/ none / none | 9.16   | 9.50        |   -3.59%   |   0.98%    |   1.56%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_3                          | parquet
/ none / none | 1.44   | 1.50        |   -4.14%   |   1.10%    |   1.08%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_filter_string_non_selective                  | parquet
/ none / none | 0.89   | 0.93        |   -4.31%   |   7.20%    |   5.12%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_decimal_highndv                      | parquet
/ none / none | 13.98  | 14.94       |   -6.40%   | * 11.24% * |   8.66%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q2                                            | parquet
/ none / none | 2.87   | 3.13        |   -8.36%   |   0.57%    |   1.59%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_bigint_lowndv                        | parquet
/ none / none | 1.48   | 1.62        |   -8.91%   |   1.62%    |   1.28%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_conjunct_ordering_2                          | parquet
/ none / none | 13.68  | 15.05       |   -9.14%   |   2.91%    |   1.83%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | primitive_groupby_decimal_lowndv.test                  | parquet
/ none / none | 1.47   | 1.61        |   -9.26%   |   0.19%    |   1.50%        | 1      
    | 10    |
    | TARGETED-PERF(_20) | PERF_AGG-Q3                                            | parquet
/ none / none | 4.25   | 4.82        |   -11.82%  |   0.79%    |   0.84%        | 1      
    | 10    |
    +--------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5
---
M be/src/exec/hash-table-test.cc
M be/src/exec/hash-table.cc
M be/src/exec/hash-table.h
M be/src/exec/hash-table.inline.h
M be/src/exec/partitioned-aggregation-node-ir.cc
M be/src/exec/partitioned-aggregation-node.cc
M be/src/exec/partitioned-hash-join-node-ir.cc
M be/src/exec/partitioned-hash-join-node.cc
8 files changed, 270 insertions(+), 279 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/26/4326/5
-- 
To view, visit http://gerrit.cloudera.org:8080/4326
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ie353666dbb5c958f0094d169306fe930ec3014c5
Gerrit-PatchSet: 5
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Michael Ho
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message