hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17626) Query reoptimization using cached runtime statistics
Date Tue, 06 Mar 2018 12:22:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387688#comment-16387688
] 

Hive QA commented on HIVE-17626:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 18s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 43s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 52s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  3m 29s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 15s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  8s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 23s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 52s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 52s{color} | {color:green}
the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 14s{color}
| {color:green} The patch common passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  2m  1s{color} | {color:red}
root: The patch generated 85 new + 1956 unchanged - 168 fixed = 2041 total (was 2124) {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 10s{color}
| {color:green} druid-handler: The patch generated 0 new + 1 unchanged - 1 fixed = 1 total
(was 2) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 13s{color}
| {color:green} itests/util: The patch generated 0 new + 103 unchanged - 1 fixed = 103 total
(was 104) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 51s{color} | {color:red}
ql: The patch generated 84 new + 1424 unchanged - 164 fixed = 1508 total (was 1588) {color}
|
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 13s{color} | {color:red}
standalone-metastore: The patch generated 1 new + 3 unchanged - 2 fixed = 4 total (was 5)
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  1s{color} | {color:green}
The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m  5s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 13s{color} | {color:red}
The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 39s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  xml  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03)
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9508/dev-support/hive-personality.sh
|
| git revision | master / 8d88cfa |
| Default Java | 1.8.0_111 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus/diff-checkstyle-root.txt
|
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus/diff-checkstyle-ql.txt
|
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus/whitespace-eol.txt
|
| asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus/patch-asflicense-problems.txt
|
| modules | C: common . druid-handler itests itests/util ql standalone-metastore U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9508/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> Query reoptimization using cached runtime statistics
> ----------------------------------------------------
>
>                 Key: HIVE-17626
>                 URL: https://issues.apache.org/jira/browse/HIVE-17626
>             Project: Hive
>          Issue Type: New Feature
>          Components: Logical Optimizer
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Zoltan Haindrich
>            Priority: Major
>         Attachments: HIVE-17626.01.patch, HIVE-17626.01wip01.patch, HIVE-17626.02.patch,
HIVE-17626.03.patch, HIVE-17626.04.patch, HIVE-17626.05.patch, HIVE-17626.06.patch, HIVE-17626.07A.patch,
HIVE-17626.07B.patch, HIVE-17626.08.patch, HIVE-17626.09.patch, HIVE-17626.10.patch, HIVE-17626.11.patch,
runtimestats.patch
>
>
> Something similar to "EXPLAIN ANALYZE" where we annotate explain plan with actual and
estimated statistics. The runtime stats can be cached at query level and subsequent execution
of the same query can make use of the cached statistics from the previous run for better optimization.

> Some use cases,
> 1) re-planning join query (mapjoin failures can be converted to shuffle joins)
> 2) better statistics for table scan operator if dynamic partition pruning is involved
> 3) Better estimates for bloom filter initialization (setting expected entries during
merge)
> This can extended to support wider queries by caching fragments of operator plans scanning
same table(s) or matching some operator sequences.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message