impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Ho (Code Review)" <ger...@cloudera.org>
Subject [Impala-CR](cdh5-trunk) IMPALA-3115: Hoist some variables out of loops in cross-compiled code.
Date Wed, 06 Apr 2016 22:35:22 GMT
Michael Ho has posted comments on this change.

Change subject: IMPALA-3115: Hoist some variables out of loops in cross-compiled code.
......................................................................


Patch Set 2:

(3 comments)

+----------+-----------------------+---------+------------+------------+----------------+
| Workload | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean) |
+----------+-----------------------+---------+------------+------------+----------------+
| TPCH(15) | parquet / none / none | 15.82   | -0.44%     | 9.84       | -0.47%         |
+----------+-----------------------+---------+------------+------------+----------------+

+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload | Query    | File Format           | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%)
| Base StdDev(%) | Num Clients | Iters |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TPCH(15) | TPCH-Q12 | parquet / none / none | 3.63   | 3.57        |   +1.73%   |   1.40%
  |   1.17%        | 1           | 5     |
| TPCH(15) | TPCH-Q6  | parquet / none / none | 2.55   | 2.54        |   +0.47%   |   2.93%
  |   3.15%        | 1           | 5     |
| TPCH(15) | TPCH-Q22 | parquet / none / none | 5.89   | 5.88        |   +0.29%   |   1.53%
  |   0.71%        | 1           | 5     |
| TPCH(15) | TPCH-Q7  | parquet / none / none | 28.10  | 28.02       |   +0.28%   |   1.92%
  |   0.69%        | 1           | 5     |
| TPCH(15) | TPCH-Q10 | parquet / none / none | 6.85   | 6.84        |   +0.18%   |   0.89%
  |   0.93%        | 1           | 5     |
| TPCH(15) | TPCH-Q11 | parquet / none / none | 1.86   | 1.86        |   +0.12%   |   1.27%
  |   2.54%        | 1           | 5     |
| TPCH(15) | TPCH-Q17 | parquet / none / none | 30.68  | 30.67       |   +0.01%   |   0.56%
  |   0.56%        | 1           | 5     |
| TPCH(15) | TPCH-Q9  | parquet / none / none | 40.85  | 40.85       |   -0.00%   |   0.92%
  |   0.79%        | 1           | 5     |
| TPCH(15) | TPCH-Q5  | parquet / none / none | 19.37  | 19.42       |   -0.23%   |   0.65%
  |   0.65%        | 1           | 5     |
| TPCH(15) | TPCH-Q14 | parquet / none / none | 3.31   | 3.32        |   -0.37%   |   2.45%
  |   3.42%        | 1           | 5     |
| TPCH(15) | TPCH-Q19 | parquet / none / none | 37.52  | 37.67       |   -0.40%   |   1.17%
  |   1.93%        | 1           | 5     |
| TPCH(15) | TPCH-Q21 | parquet / none / none | 32.12  | 32.28       |   -0.50%   |   4.91%
  |   2.99%        | 1           | 5     |
| TPCH(15) | TPCH-Q13 | parquet / none / none | 15.59  | 15.67       |   -0.54%   |   0.39%
  |   0.14%        | 1           | 5     |
| TPCH(15) | TPCH-Q18 | parquet / none / none | 35.04  | 35.26       |   -0.61%   |   0.83%
  |   0.71%        | 1           | 5     |
| TPCH(15) | TPCH-Q2  | parquet / none / none | 2.68   | 2.70        |   -0.93%   |   1.64%
  |   2.03%        | 1           | 5     |
| TPCH(15) | TPCH-Q15 | parquet / none / none | 5.04   | 5.09        |   -0.97%   |   1.30%
  |   3.00%        | 1           | 5     |
| TPCH(15) | TPCH-Q8  | parquet / none / none | 34.95  | 35.32       |   -1.05%   |   0.70%
  |   0.41%        | 1           | 5     |
| TPCH(15) | TPCH-Q3  | parquet / none / none | 12.05  | 12.19       |   -1.14%   |   0.36%
  |   0.80%        | 1           | 5     |
| TPCH(15) | TPCH-Q4  | parquet / none / none | 14.08  | 14.24       |   -1.14%   |   0.37%
  |   0.93%        | 1           | 5     |
| TPCH(15) | TPCH-Q20 | parquet / none / none | 6.55   | 6.65        |   -1.50%   |   1.38%
  |   1.11%        | 1           | 5     |
| TPCH(15) | TPCH-Q16 | parquet / none / none | 1.68   | 1.71        |   -1.55%   |   2.15%
  |   1.39%        | 1           | 5     |
| TPCH(15) | TPCH-Q1  | parquet / none / none | 7.73   | 7.92        |   -2.39%   |   0.73%
  |   1.37%        | 1           | 5     |
+----------+----------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+

+-------------------+-----------------------+---------+------------+------------+----------------+
| Workload          | File Format           | Avg (s) | Delta(Avg) | GeoMean(s) | Delta(GeoMean)
|
+-------------------+-----------------------+---------+------------+------------+----------------+
| TARGETED-PERF(15) | parquet / none / none | 7.68    | -1.10%     | 2.94       | -1.13% 
       |
+-------------------+-----------------------+---------+------------+------------+----------------+

+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| Workload          | Query                                                  | File Format
          | Avg(s) | Base Avg(s) | Delta(Avg) | StdDev(%) | Base StdDev(%) | Num Clients |
Iters |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+
| TARGETED-PERF(15) | PERF_STRING-Q5                                         | parquet / none
/ none | 2.05   | 2.00        |   +2.92%   |   3.64%   |   1.08%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_bigint_non_selective                  | parquet / none
/ none | 1.08   | 1.05        |   +2.71%   |   2.04%   |   1.93%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q1                                         | parquet / none
/ none | 1.61   | 1.58        |   +1.85%   |   2.22%   |   2.78%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q7                                         | parquet / none
/ none | 3.06   | 3.01        |   +1.66%   |   2.44%   |   1.69%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_orderby_all                                  | parquet / none
/ none | 10.42  | 10.29       |   +1.21%   |   1.46%   |   0.83%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q1                                            | parquet / none
/ none | 1.42   | 1.40        |   +1.11%   |   4.51%   |   1.70%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q7                                            | parquet / none
/ none | 1.19   | 1.17        |   +0.99%   |   4.28%   |   2.77%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_shuffle_join_one_to_many_string_with_groupby | parquet / none
/ none | 73.02  | 72.31       |   +0.98%   |   0.18%   |   0.46%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q3                                         | parquet / none
/ none | 1.80   | 1.79        |   +0.89%   |   2.38%   |   1.39%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_broadcast_join_3                             | parquet / none
/ none | 3.24   | 3.21        |   +0.85%   |   1.26%   |   2.06%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_LIMIT-Q1                                          | parquet / none
/ none | 0.01   | 0.01        |   +0.45%   |   3.89%   |   1.12%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q2                                         | parquet / none
/ none | 1.71   | 1.71        |   +0.33%   |   2.03%   |   3.58%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_broadcast_join_2                             | parquet / none
/ none | 2.08   | 2.08        |   +0.16%   |   2.03%   |   1.06%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_decimal_selective                     | parquet / none
/ none | 1.17   | 1.17        |   +0.14%   |   2.11%   |   2.14%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_broadcast_join_1                             | parquet / none
/ none | 1.11   | 1.11        |   +0.13%   |   3.80%   |   5.04%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_string_non_selective                  | parquet / none
/ none | 1.49   | 1.49        |   -0.07%   |   4.10%   |   9.80%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_exchange_broadcast                           | parquet / none
/ none | 14.81  | 14.83       |   -0.09%   |   0.62%   |   0.69%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_orderby_bigint                               | parquet / none
/ none | 2.50   | 2.51        |   -0.21%   |   3.08%   |   3.73%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q5                                            | parquet / none
/ none | 2.45   | 2.47        |   -0.59%   |   2.45%   |   2.20%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_groupby_bigint_pk                            | parquet / none
/ none | 29.22  | 29.44       |   -0.75%   |   0.87%   |   1.05%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q4                                         | parquet / none
/ none | 1.79   | 1.81        |   -0.94%   |   3.16%   |   3.40%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_bigint_selective                      | parquet / none
/ none | 0.99   | 1.00        |   -0.94%   |   3.73%   |   2.13%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_string_selective                      | parquet / none
/ none | 1.17   | 1.19        |   -1.04%   |   3.63%   |   6.77%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q6                                            | parquet / none
/ none | 1.12   | 1.14        |   -1.11%   |   3.77%   |   0.11%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_string_like                           | parquet / none
/ none | 5.18   | 5.24        |   -1.15%   |   1.04%   |   1.46%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_topn_bigint                                  | parquet / none
/ none | 5.51   | 5.57        |   -1.15%   |   6.09%   |   9.13%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_top-n_all                                    | parquet / none
/ none | 31.54  | 32.08       |   -1.68%   |   0.82%   |   0.64%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_shuffle_join_union_all_with_groupby          | parquet / none
/ none | 19.42  | 19.77       |   -1.77%   |   0.66%   |   0.61%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_filter_decimal_non_selective                 | parquet / none
/ none | 1.23   | 1.25        |   -1.88%   |   1.89%   |   1.70%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_groupby_bigint_highndv                       | parquet / none
/ none | 8.70   | 8.95        |   -2.80%   |   0.92%   |   0.40%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_STRING-Q6                                         | parquet / none
/ none | 3.71   | 3.82        |   -2.96%   |   1.16%   |   3.03%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_groupby_decimal_highndv                      | parquet / none
/ none | 14.92  | 15.40       |   -3.09%   |   0.28%   |   0.74%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q3                                            | parquet / none
/ none | 3.86   | 4.00        |   -3.45%   |   0.56%   |   0.62%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_exchange_shuffle                             | parquet / none
/ none | 25.77  | 26.70       |   -3.49%   |   0.29%   |   0.26%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q2                                            | parquet / none
/ none | 2.44   | 2.56        |   -4.86%   |   1.02%   |   0.02%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_groupby_bigint_lowndv                        | parquet / none
/ none | 1.49   | 1.59        |   -6.26%   |   3.68%   |   2.74%        | 1           | 5
    |
| TARGETED-PERF(15) | primitive_groupby_decimal_lowndv.test                  | parquet / none
/ none | 1.44   | 1.55        |   -7.27%   |   3.01%   |   1.41%        | 1           | 5
    |
| TARGETED-PERF(15) | PERF_AGG-Q4                                            | parquet / none
/ none | 6.24   | 6.98        |   -10.63%  |   7.27%   |   5.67%        | 1           | 5
    |
+-------------------+--------------------------------------------------------+-----------------------+--------+-------------+------------+-----------+----------------+-------------+-------+

http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/exec/partitioned-aggregation-node-ir.cc
File be/src/exec/partitioned-aggregation-node-ir.cc:

Line 195:     const HashTableCtx* __restrict__ ht_ctx, Partition* __restrict__ partition,
> I think these restrict qualifiers may be discarded when TryAddToHashTable i
Yes. I verified the IR is the same without the qualifier.


http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/exec/partitioned-hash-join-node-ir.cc
File be/src/exec/partitioned-hash-join-node-ir.cc:

Line 274: buildStatus_
> while you're here, make it build_status_?
Done


http://gerrit.cloudera.org:8080/#/c/2661/2/be/src/runtime/row-batch.h
File be/src/runtime/row-batch.h:

Line 145:   /// For row batches with zero capacity, Next() will always return NULL.
> Maybe we should just DCHECK if Next() is called on an empty batch? The code
DCHECK_LT(row_idx, capacity) was actually triggered in GetRow() during testing but I couldn't
recover a proper backtrace as I couldn't find the binary in the AWS VM. The DCHECK message
suggested that both num_rows_ and capacity_ are zero so that seems to be a legitimate case.


-- 
To view, visit http://gerrit.cloudera.org:8080/2661
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Ic7152b1fb094b3c3574d203e3774f4297f2225dc
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Michael Ho <kwho@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstrong@cloudera.com>
Gerrit-HasComments: Yes

Mime
View raw message