hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-3959) Update Partition Statistics in Metastore Layer
Date Fri, 25 Oct 2013 22:44:30 GMT

    [ https://issues.apache.org/jira/browse/HIVE-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13805775#comment-13805775
] 

Hive QA commented on HIVE-3959:
-------------------------------



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12610142/HIVE-3959.4.patch

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 4481 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_create_like_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_join_star
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_pcr
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_union_view
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1237/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1237/console

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests failed with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

> Update Partition Statistics in Metastore Layer
> ----------------------------------------------
>
>                 Key: HIVE-3959
>                 URL: https://issues.apache.org/jira/browse/HIVE-3959
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore, Statistics
>            Reporter: Bhushan Mandhani
>            Assignee: Ashutosh Chauhan
>            Priority: Minor
>         Attachments: HIVE-3959.1.patch, HIVE-3959.2.patch, HIVE-3959.3.patch, HIVE-3959.4.patch,
HIVE-3959.patch.1, HIVE-3959.patch.11.txt, HIVE-3959.patch.12.txt, HIVE-3959.patch.2
>
>
> When partitions are created using queries ("insert overwrite" and "insert into") then
the StatsTask updates all stats. However, when partitions are added directly through metadata-only
partitions (either CLI or direct calls to Thrift Metastore) no stats are populated even if
hive.stats.reliable is set to true. This puts us in a situation where we can't decide if stats
are truly reliable or not.
> We propose that the "fast stats" (numFiles and totalSize) which don't require a scan
of the data should always be populated and be completely reliable. For now we are still excluding
rowCount and rawDataSize because that will make these operations very expensive. Currently
they are quick metadata-only ops.  



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message