quickstep-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jianqiao <...@git.apache.org>
Subject [GitHub] incubator-quickstep issue #300: QUICKSTEP-106: Hash-Join-Fuse: Feature added...
Date Wed, 22 Nov 2017 07:04:27 GMT
Github user jianqiao commented on the issue:

    @dylanpbacon Hi Dylan, I updated the LIP related stuff and put it in this [branch](https://github.com/apache/incubator-quickstep/tree/Hash-Join-Fuse),
you may just fetch the changes to reset your repo's `Hash-Join-Fuse` branch, then `git push
-f`, then we can merge this PR.
    The optimization is enabled by default, and I added an extra gflag `fuse-hash-select-threshold`
that is set to one million (`1000000u`) by default. A fusion transformation is applied only
when the estimated cardinality of the build-side selection is _greater_ than the threshold.
    Overall speaking, the fuse-hash-select optimization is especially beneficial when _the
build-side selection has large output cardinality_ (e.g. TPC-H Q21), and the benefits come
from two aspects:
    (1) smaller memory footprint,
    (2) avoiding materialization of the selection's output.
    However, due to some issues in current implementation of HashJoinOperator (and buffer
manager), the fusion may slow down some queries (e.g. TPC-H Q02, 400ms -> 700ms with LIP
fixed). The `fuse-hash-select-threshold` prevents those situations.


View raw message