hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20246) Configurable collecting stats by using DO_NOT_UPDATE_STATS table property
Date Thu, 16 Aug 2018 14:31:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582599#comment-16582599
] 

Hive QA commented on HIVE-20246:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12935784/HIVE-20246.7.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14885 tests passed

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/13268/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/13268/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-13268/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12935784 - PreCommit-HIVE-Build

> Configurable collecting stats by using DO_NOT_UPDATE_STATS table property
> -------------------------------------------------------------------------
>
>                 Key: HIVE-20246
>                 URL: https://issues.apache.org/jira/browse/HIVE-20246
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>            Reporter: Alice Fan
>            Assignee: Alice Fan
>            Priority: Minor
>             Fix For: 4.0.0
>
>         Attachments: HIVE-20246.5.patch, HIVE-20246.6.patch, HIVE-20246.7.patch
>
>
> By default, Hive collects stats when running operations like alter table partition(s),
create table, and create external table. However, collecting stats requires Metastore lists
all files under the table directory and the file listing operation can be very expensive particularly
on filesystems like S3.
> HIVE-18743 made DO_NOT_UPDATE_STATS table property could be selectively prevent stats
collection. 
> This Jira aims at introducing DO_NOT_UPDATE_STATS table property into the MetaStoreUtils.updatePartitionStatsFast.
By adding this, user can be selectively prevent stats collection when doing alter table partition(s)
operation at table level. For example, set 'Alter Table S3_Table set tblproperties('DO_NOT_UPDATE_STATS'='TRUE');'
MetaStore will not collect stats for the specified S3_Table when alter table add partition(key1=val1,
key2=val2);



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message