impala-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Armstrong (Code Review)" <ger...@cloudera.org>
Subject [Impala-ASF-CR] IMPALA-5860: upgrade to LLVM 3.9.1
Date Sat, 16 Sep 2017 21:43:28 GMT
Hello Dan Hecht,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/7974

to look at the new patch set (#7).

Change subject: IMPALA-5860: upgrade to LLVM 3.9.1
......................................................................

IMPALA-5860: upgrade to LLVM 3.9.1

LLVM made a few API changes:
* Misc minor changes to function and type signatures
* The CloneFunction() API changed semantics (http://reviews.llvm.org/D18628)

Testing:
Ran core and ASAN tests.

Perf:
Ran single node TPC-H and targeted perf with scale factor 60. Both
improved on average.

+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(60) | parquet / none / none | 17.82   | -5.01%     | 11.64      | -4.23%         |
+----------+-----------------------+---------+------------+------------+----------------+

+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload | Query    | File Format           | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)
 | Base StdDev(%) | Num Clients | Iters |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TPCH(60) | TPCH-Q1  | parquet / none / none | 27.97  | 27.59       |   +1.36%   |   0.39%
   |   0.41%        | 1           | 5     |
| TPCH(60) | TPCH-Q20 | parquet / none / none | 5.81   | 5.78        |   +0.44%   |   0.73%
   |   0.21%        | 1           | 5     |
| TPCH(60) | TPCH-Q21 | parquet / none / none | 62.98  | 62.98       |   +0.01%   |   5.56%
   |   1.07%        | 1           | 5     |
| TPCH(60) | TPCH-Q15 | parquet / none / none | 8.45   | 8.46        |   -0.20%   |   0.40%
   |   0.38%        | 1           | 5     |
| TPCH(60) | TPCH-Q4  | parquet / none / none | 5.57   | 5.59        |   -0.41%   |   0.43%
   |   0.80%        | 1           | 5     |
| TPCH(60) | TPCH-Q6  | parquet / none / none | 3.16   | 3.17        |   -0.45%   |   0.78%
   |   1.70%        | 1           | 5     |
| TPCH(60) | TPCH-Q5  | parquet / none / none | 7.41   | 7.47        |   -0.92%   |   0.71%
   |   1.06%        | 1           | 5     |
| TPCH(60) | TPCH-Q9  | parquet / none / none | 33.45  | 33.78       |   -0.99%   |   1.15%
   |   0.85%        | 1           | 5     |
| TPCH(60) | TPCH-Q11 | parquet / none / none | 2.00   | 2.03        |   -1.34%   |   1.71%
   |   2.24%        | 1           | 5     |
| TPCH(60) | TPCH-Q2  | parquet / none / none | 4.71   | 4.79        |   -1.60%   |   1.49%
   |   1.95%        | 1           | 5     |
| TPCH(60) | TPCH-Q18 | parquet / none / none | 46.48  | 47.71       |   -2.58%   |   1.04%
   |   0.38%        | 1           | 5     |
| TPCH(60) | TPCH-Q14 | parquet / none / none | 5.85   | 6.02        |   -2.84%   |   0.44%
   |   0.70%        | 1           | 5     |
| TPCH(60) | TPCH-Q22 | parquet / none / none | 6.51   | 6.76        |   -3.71%   |   2.29%
   |   2.42%        | 1           | 5     |
| TPCH(60) | TPCH-Q19 | parquet / none / none | 7.27   | 7.63        |   -4.69%   |   1.33%
   |   0.78%        | 1           | 5     |
| TPCH(60) | TPCH-Q10 | parquet / none / none | 13.19  | 13.84       |   -4.73%   |   0.42%
   |   1.44%        | 1           | 5     |
| TPCH(60) | TPCH-Q13 | parquet / none / none | 21.95  | 23.12       |   -5.03%   |   0.25%
   |   1.19%        | 1           | 5     |
| TPCH(60) | TPCH-Q16 | parquet / none / none | 5.29   | 5.57        |   -5.04%   |   0.85%
   |   0.78%        | 1           | 5     |
| TPCH(60) | TPCH-Q7  | parquet / none / none | 42.05  | 44.33       |   -5.16%   |   2.07%
   |   2.28%        | 1           | 5     |
| TPCH(60) | TPCH-Q12 | parquet / none / none | 19.77  | 21.00       |   -5.87%   |   8.14%
   |   5.09%        | 1           | 5     |
| TPCH(60) | TPCH-Q3  | parquet / none / none | 11.46  | 12.32       |   -6.94%   |   0.76%
   |   0.53%        | 1           | 5     |
| TPCH(60) | TPCH-Q17 | parquet / none / none | 40.09  | 49.28       |   -18.64%  |   2.09%
   |   0.67%        | 1           | 5     |
| TPCH(60) | TPCH-Q8  | parquet / none / none | 10.63  | 13.47       | I -21.08%  | * 12.34%
* | * 21.09% *     | 1           | 5     |
+----------+----------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload          | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean)
|
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(60) | parquet / none / none | 22.38   | -1.24%     | 4.17       | +0.81% 
       |
+-------------------+-----------------------+---------+------------+------------+----------------+

+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| Workload          | Query                                                  | File Format
          | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)  | Base StdDev(%) | Num Clients
| Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+
| TARGETED-PERF(60) | primitive_conjunct_ordering_1                          | parquet / none
/ none | 0.12   | 0.10        | R +22.38%  |   0.81%    | * 27.26% *     | 1           | 5
    |
| TARGETED-PERF(60) | primitive_groupby_decimal_highndv                      | parquet / none
/ none | 29.86  | 25.46       |   +17.31%  |   6.18%    |   3.83%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_LIMIT-Q1                                          | parquet / none
/ none | 0.01   | 0.01        |   +13.41%  | * 15.35% * |   2.95%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_bigint_non_selective                  | parquet / none
/ none | 0.88   | 0.82        |   +7.17%   |   9.52%    |   3.59%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_decimal_non_selective                 | parquet / none
/ none | 1.48   | 1.41        |   +4.94%   |   4.23%    |   1.86%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_small_join_1                                 | parquet / none
/ none | 0.18   | 0.18        |   +4.26%   | * 11.92% * |   2.43%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_broadcast_join_3                             | parquet / none
/ none | 7.29   | 7.03        |   +3.77%   |   5.98%    |   9.35%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_exchange_broadcast                           | parquet / none
/ none | 38.41  | 37.02       |   +3.77%   |   8.59%    |   1.31%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q6                                            | parquet / none
/ none | 1.93   | 1.89        |   +2.14%   |   2.22%    |   1.75%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_conjunct_ordering_2                          | parquet / none
/ none | 7.26   | 7.17        |   +1.29%   |   2.28%    |   4.54%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q1                                         | parquet / none
/ none | 2.79   | 2.75        |   +1.28%   |   0.52%    |   0.76%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q3                                         | parquet / none
/ none | 3.51   | 3.47        |   +1.01%   |   0.63%    |   0.57%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_string_selective                      | parquet / none
/ none | 1.05   | 1.04        |   +0.76%   |   3.03%    |   2.40%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_orderby_bigint                               | parquet / none
/ none | 4.88   | 4.84        |   +0.75%   |   0.58%    |   0.97%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_top-n_all                                    | parquet / none
/ none | 38.56  | 38.28       |   +0.73%   |   0.20%    |   0.24%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_orderby_all                                  | parquet / none
/ none | 25.68  | 25.54       |   +0.55%   |   0.27%    |   0.40%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_shuffle_join_union_all_with_groupby          | parquet / none
/ none | 54.02  | 53.74       |   +0.53%   |   0.35%    |   0.23%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q5                                         | parquet / none
/ none | 4.28   | 4.26        |   +0.43%   |   0.68%    |   0.47%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_empty_build_join_1                           | parquet / none
/ none | 16.25  | 16.19       |   +0.42%   |   0.33%    |   0.42%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_groupby_bigint_highndv                       | parquet / none
/ none | 32.49  | 32.36       |   +0.42%   |   0.23%    |   0.88%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q1                                            | parquet / none
/ none | 2.22   | 2.21        |   +0.34%   |   1.82%    |   1.88%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_groupby_bigint_pk                            | parquet / none
/ none | 112.73 | 112.50      |   +0.21%   |   0.75%    |   0.99%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q4                                         | parquet / none
/ none | 3.52   | 3.51        |   +0.13%   |   0.58%    |   0.65%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q2                                         | parquet / none
/ none | 3.06   | 3.06        |   +0.03%   |   0.69%    |   0.76%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_decimal_selective                     | parquet / none
/ none | 1.20   | 1.20        |   -0.01%   |   2.35%    |   1.24%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_broadcast_join_2                             | parquet / none
/ none | 4.27   | 4.27        |   -0.03%   |   0.52%    |   0.48%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_groupby_decimal_lowndv.test                  | parquet / none
/ none | 3.87   | 3.87        |   -0.07%   |   1.69%    |   1.63%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q7                                            | parquet / none
/ none | 1.92   | 1.93        |   -0.28%   |   2.33%    |   1.94%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q5                                            | parquet / none
/ none | 0.48   | 0.48        |   -0.28%   |   0.59%    |   0.53%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q4                                            | parquet / none
/ none | 17.48  | 17.53       |   -0.30%   |   0.43%    |   0.58%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q7                                         | parquet / none
/ none | 7.87   | 7.90        |   -0.35%   |   0.67%    |   0.55%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_exchange_shuffle                             | parquet / none
/ none | 74.25  | 74.53       |   -0.37%   |   0.57%    |   0.36%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_groupby_bigint_lowndv                        | parquet / none
/ none | 3.81   | 3.82        |   -0.42%   |   1.51%    |   1.10%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q2                                            | parquet / none
/ none | 9.93   | 10.00       |   -0.67%   |   0.77%    |   0.67%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_string_like                           | parquet / none
/ none | 14.63  | 14.74       |   -0.72%   |   0.24%    |   0.02%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_conjunct_ordering_4                          | parquet / none
/ none | 0.23   | 0.23        |   -0.82%   |   0.59%    |   1.31%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_STRING-Q6                                         | parquet / none
/ none | 9.87   | 10.03       |   -1.55%   |   0.39%    |   0.22%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none
/ none | 262.13 | 268.18      |   -2.26%   |   0.31%    |   0.27%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_string_non_selective                  | parquet / none
/ none | 1.23   | 1.26        |   -2.26%   |   1.72%    |   2.15%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_broadcast_join_1                             | parquet / none
/ none | 2.04   | 2.09        |   -2.54%   |   0.31%    |   2.88%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_conjunct_ordering_3                          | parquet / none
/ none | 0.13   | 0.13        |   -3.13%   |   0.73%    |   2.50%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_bigint_selective                      | parquet / none
/ none | 0.12   | 0.12        |   -3.15%   |   1.03%    |   1.73%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_conjunct_ordering_5                          | parquet / none
/ none | 14.11  | 14.60       |   -3.33%   |   2.03%    |   2.43%        | 1           | 5
    |
| TARGETED-PERF(60) | PERF_AGG-Q3                                            | parquet / none
/ none | 8.28   | 8.64        |   -4.17%   |   0.79%    |   1.08%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_long_predicate                               | parquet / none
/ none | 215.27 | 227.90      |   -5.54%   |   0.06%    |   0.08%        | 1           | 5
    |
| TARGETED-PERF(60) | primitive_topn_bigint                                  | parquet / none
/ none | 4.48   | 4.81        |   -6.90%   |   8.50%    | * 15.79% *     | 1           | 5
    |
| TARGETED-PERF(60) | primitive_filter_in_predicate                          | parquet / none
/ none | 1.84   | 1.99        |   -7.51%   |   3.98%    |   5.29%        | 1           | 5
    |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+------------+----------------+-------------+-------+

Change-Id: Ida873ddb15e393b0bd37486db24add8a32f43ad0
---
M be/src/codegen/codegen-symbol-emitter.cc
M be/src/codegen/instruction-counter-test.cc
M be/src/codegen/llvm-codegen.cc
M be/src/codegen/llvm-codegen.h
M be/src/common/compiler-util.h
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
7 files changed, 31 insertions(+), 25 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/7974/7
-- 
To view, visit http://gerrit.cloudera.org:8080/7974
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ida873ddb15e393b0bd37486db24add8a32f43ad0
Gerrit-PatchSet: 7
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhecht@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>

Mime
View raw message