hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8955) alter partition should check for "hive.stats.autogather" in hiveConf
Date Mon, 24 Nov 2014 22:38:13 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14223674#comment-14223674
] 

Szehon Ho commented on HIVE-8955:
---------------------------------

Hi Pankit, I took a look.  It seems like for alter table/partition case, there's different
flag being checked than hive.stats.autogather, which says "A flag to gather statistics automatically
during the INSERT OVERWRITE command."

The stats do seem to be correctly updated though at [here|https://github.com/apache/hive/blob/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java#L215]
and [here|https://github.com/apache/hive/blob/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java#L461]
as per my limited understanding.

> alter partition should check for "hive.stats.autogather" in hiveConf
> --------------------------------------------------------------------
>
>                 Key: HIVE-8955
>                 URL: https://issues.apache.org/jira/browse/HIVE-8955
>             Project: Hive
>          Issue Type: Improvement
>          Components: Metastore
>    Affects Versions: 0.13.1
>            Reporter: Pankit Thapar
>             Fix For: 0.15.0
>
>
> When alter partition code path is triggered, it should check for the flag "hive.stats.autogather",
if it is true, then only updateStats else skip them.
> This is done in append_partition code flow. 
> Is there any specific reason the alter_partition does not respect this conf variable?
> //code snippet : HiveMetastore.java 
>  private Partition append_partition_common(RawStore ms, String dbName, String tableName,
>         List<String> part_vals, EnvironmentContext envContext) throws InvalidObjectException,
>         AlreadyExistsException, MetaException {
> ...
> ....
>         if (HiveConf.getBoolVar(hiveConf, HiveConf.ConfVars.HIVESTATSAUTOGATHER) &&
>             !MetaStoreUtils.isView(tbl)) {
>           MetaStoreUtils.updatePartitionStatsFast(part, wh, madeDir);
>         }
> ...
> ...
> }
> The above code snippet checks for the variable but this same check is absent in 
> //code snippet : HiveAlterHandler.java 
> public Partition alterPartition(final RawStore msdb, Warehouse wh, final String dbname,
>       final String name, final List<String> part_vals, final Partition new_part)
>       throws InvalidOperationException, InvalidObjectException, AlreadyExistsException,
>       MetaException {
> ....
> ...
> }



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message