hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16654) Optimize a combination of avg(), sum(), count(distinct) etc
Date Sun, 21 May 2017 05:46:04 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16018700#comment-16018700
] 

Hive QA commented on HIVE-16654:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12869152/HIVE-16654.01.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 68 failed/errored test(s), 10743 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby3_map] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby3_map_skew] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_11] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_8] (batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup4] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_skew] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin10] (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin11] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin1] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin2] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin3] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin4] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin6] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin7] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_mapjoin9] (batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_union_remove_1] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_union_remove_2] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt10] (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt11] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt12] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt14] (batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt16] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt17] (batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt19] (batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt1] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt20] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt21] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt3] (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt4] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt5] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt6] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt7] (batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoinopt8] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_count] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_empty_where] (batchId=22)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[count_dist_rewrite] (batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_1]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[metadataonly1] (batchId=156)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query16] (batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query28] (batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query94] (batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query95] (batchId=231)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby3_map] (batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby3_map_skew] (batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[nullgroup4] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin_union_remove_1] (batchId=136)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoin_union_remove_2] (batchId=111)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt10] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt11] (batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt12] (batchId=103)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt14] (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt15] (batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt16] (batchId=105)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt17] (batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt19] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt1] (batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt20] (batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt2] (batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt3] (batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt4] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt5] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt6] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt7] (batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[skewjoinopt8] (batchId=111)
org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteTimestamp (batchId=179)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDate (batchId=179)
org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteSmallint (batchId=179)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5372/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5372/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5372/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 68 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12869152 - PreCommit-HIVE-Build

> Optimize a combination of avg(), sum(), count(distinct) etc
> -----------------------------------------------------------
>
>                 Key: HIVE-16654
>                 URL: https://issues.apache.org/jira/browse/HIVE-16654
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Pengcheng Xiong
>            Assignee: Pengcheng Xiong
>         Attachments: HIVE-16654.01.patch
>
>
> an example rewrite for q28 of tpcds is 
> {code}
> (select LP as B1_LP ,CNT  as B1_CNT,CNTD as B1_CNTD
>       from (select sum(xc0) / sum(xc1) as LP, sum(xc1) as CNT, count(1) as CNTD from
(select sum(ss_list_price) as xc0, count(ss_list_price) as xc1 from store_sales  where 
> ss_list_price is not null and ss_quantity between 0 and 5
>         and (ss_list_price between 11 and 11+10 
>              or ss_coupon_amt between 460 and 460+1000
>              or ss_wholesale_cost between 14 and 14+20)
>  group by ss_list_price) ss0) ss1) B1
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message