hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17421) Clear incorrect stats after replication
Date Fri, 01 Sep 2017 02:18:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16149923#comment-16149923
] 

Hive QA commented on HIVE-17421:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12884794/HIVE-17421.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11022 tests executed
*Failed tests:*
{noformat}
TestTxnCommandsBase - did not produce a TEST-*.xml file (likely timed out) (batchId=280)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=61)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=143)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=100)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=234)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/6628/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/6628/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-6628/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12884794 - PreCommit-HIVE-Build

> Clear incorrect stats after replication
> ---------------------------------------
>
>                 Key: HIVE-17421
>                 URL: https://issues.apache.org/jira/browse/HIVE-17421
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>         Attachments: HIVE-17421.1.patch
>
>
> After replication, some stats summary are incorrect. If hive.compute.query.using.stats
set to true, we will get wrong result on the destination side.
> This will not happen with bootstrap replication. This is because stats summary are in
table properties and will be replicated to the destination. However, in incremental replication,
this won't work. When creating table, the stats summary are empty (eg, numRows=0). Later when
we insert data, stats summary are updated with update_table_column_statistics/update_partition_column_statistics,
however, both events are not captured in incremental replication. Thus on the destination
side, we will get count\(*\)=0. The simple solution is to remove COLUMN_STATS_ACCURATE property
after incremental replication.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message