hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vineet Garg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17634) Use properties from HiveConf about "fetchColStats" and "fetchPartStats" in RelOptHiveTable#updateColStats
Date Fri, 29 Sep 2017 05:00:06 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16185362#comment-16185362
] 

Vineet Garg commented on HIVE-17634:
------------------------------------

Thanks for the clarification. I guess it makes sense. May be for large number of columns user
do turn this flag off and it'll be better in that case to have estimated stats instead of
no stats.

> Use properties from HiveConf about "fetchColStats" and "fetchPartStats" in RelOptHiveTable#updateColStats
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-17634
>                 URL: https://issues.apache.org/jira/browse/HIVE-17634
>             Project: Hive
>          Issue Type: Bug
>            Reporter: liyunzhang_intel
>            Assignee: liyunzhang_intel
>         Attachments: HIVE-17634.patch
>
>
> in [RelOptHiveTable#updateColStats|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java#L309],
we set {{fetchColStats}},{{fetchPartStats}} as true when call {{StatsUtils.collectStatistics}}
> {code}
>    if (!hiveTblMetadata.isPartitioned()) {
>         // 2.1 Handle the case for unpartitioned table.
>         try {
>           Statistics stats = StatsUtils.collectStatistics(hiveConf, null,
>               hiveTblMetadata, hiveNonPartitionCols, nonPartColNamesThatRqrStats,
>               colStatsCached, nonPartColNamesThatRqrStats, true, true);
>       ...
> {code}
> This will cause querying columns statistic from metastore even we set  {{hive.stats.fetch.column.stats}}
and {{hive.stats.fetch.partition.stats}} as false in HiveConf.  If we these two properties
as false, we can not any column statistics from metastore.  Suggest to set the properties
from HiveConf. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message