hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17465) Statistics: Drill-down filters don't reduce row-counts progressively
Date Tue, 12 Sep 2017 12:26:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16162855#comment-16162855
] 

Hive QA commented on HIVE-17465:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12886544/HIVE-17465.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 39 failed/errored test(s), 11037 tests executed
*Failed tests:*
{noformat}
TestAccumuloCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=230)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[flatten_and_or] (batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_multi_single_reducer2] (batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_multiskew_2] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_gby4] (batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[multi_insert_gby] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup4] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_gby2] (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_gby] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[select_unquote_or] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_include_no_sel] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_1] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_8] (batchId=45)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_insert_lateral_view]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_include_no_sel]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_1] (batchId=158)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[drop_table_failure2] (batchId=89)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=234)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_multi_single_reducer2]
(batchId=109)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[multi_insert_gby] (batchId=108)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[multi_insert_lateral_view] (batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_1] (batchId=126)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_4] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_5] (batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_6] (batchId=113)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_9] (batchId=101)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_div0] (batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_short_regress] (batchId=122)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_math_funcs] (batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_string_funcs] (batchId=125)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6782/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6782/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6782/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 39 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12886544 - PreCommit-HIVE-Build

> Statistics: Drill-down filters don't reduce row-counts progressively
> --------------------------------------------------------------------
>
>                 Key: HIVE-17465
>                 URL: https://issues.apache.org/jira/browse/HIVE-17465
>             Project: Hive
>          Issue Type: Bug
>          Components: Physical Optimizer, Statistics
>            Reporter: Gopal V
>            Assignee: Vineet Garg
>         Attachments: HIVE-17465.1.patch, HIVE-17465.2.patch
>
>
> {code}
> explain select count(d_date_sk) from date_dim where d_year=2001 ;
> explain select count(d_date_sk) from date_dim where d_year=2001  and d_moy = 9;
> explain select count(d_date_sk) from date_dim where d_year=2001 and d_moy = 9 and d_dom
= 21;
> {code}
> All 3 queries end up with the same row-count estimates after the filter.
> {code}
>             Map Operator Tree:
>                 TableScan
>                   alias: date_dim
>                   filterExpr: (d_year = 2001) (type: boolean)
>                   Statistics: Num rows: 73049 Data size: 82034027 Basic stats: COMPLETE
Column stats: COMPLETE
>                   Filter Operator
>                     predicate: (d_year = 2001) (type: boolean)
>                     Statistics: Num rows: 363 Data size: 4356 Basic stats: COMPLETE Column
stats: COMPLETE
>  
>         Map 1 
>             Map Operator Tree:
>                 TableScan
>                   alias: date_dim
>                   filterExpr: ((d_year = 2001) and (d_moy = 9)) (type: boolean)
>                   Statistics: Num rows: 73049 Data size: 82034027 Basic stats: COMPLETE
Column stats: COMPLETE
>                   Filter Operator
>                     predicate: ((d_year = 2001) and (d_moy = 9)) (type: boolean)
>                     Statistics: Num rows: 363 Data size: 5808 Basic stats: COMPLETE Column
stats: COMPLETE
>         Map 1 
>             Map Operator Tree:
>                 TableScan
>                   alias: date_dim
>                   filterExpr: ((d_year = 2001) and (d_moy = 9) and (d_dom = 21)) (type:
boolean)
>                   Statistics: Num rows: 73049 Data size: 82034027 Basic stats: COMPLETE
Column stats: COMPLETE
>                   Filter Operator
>                     predicate: ((d_year = 2001) and (d_moy = 9) and (d_dom = 21)) (type:
boolean)
>                     Statistics: Num rows: 363 Data size: 7260 Basic stats: COMPLETE Column
stats: COMPLETE
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message