impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Behm (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5011) Explain plan shows per partition stats even after dropping incremental stats
Date Wed, 26 Apr 2017 15:37:04 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Behm resolved IMPALA-5011.
------------------------------------
    Resolution: Not A Bug

> Explain plan shows per partition stats even after dropping incremental stats
> ----------------------------------------------------------------------------
>
>                 Key: IMPALA-5011
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5011
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Catalog
>    Affects Versions: Impala 2.9.0
>            Reporter: Mostafa Mokhtar
>         Attachments: table stats befor and after.zip
>
>
> All commands below were executed from the same daemon. 
> Table 100k_partitions_1m_files_8 had 5 partitions with incremental stats
> {code}
> 1000	18	10	1.19KB	NOT CACHED	NOT CACHED	TEXT	true	hdfs://vd1315.halxg.cloudera.com:8020/user/hive/warehouse/many_files.db/metrics_table/part=1000
> 1001	18	10	1.19KB	NOT CACHED	NOT CACHED	TEXT	true	hdfs://vd1315.halxg.cloudera.com:8020/user/hive/warehouse/many_files.db/metrics_table/part=1001
> 1034	18	10	1.19KB	NOT CACHED	NOT CACHED	TEXT	true	hdfs://vd1315.halxg.cloudera.com:8020/user/hive/warehouse/many_files.db/metrics_table/part=1034
> 1040	18	9	1.19KB	NOT CACHED	NOT CACHED	TEXT	true	hdfs://vd1315.halxg.cloudera.com:8020/user/hive/warehouse/many_files.db/metrics_table/part=1040
> 1041	18	9	1.19KB	NOT CACHED	NOT CACHED	TEXT	true	hdfs://vd1315.halxg.cloudera.com:8020/user/hive/warehouse/many_files.db/metrics_table/part=1041
> {code}
> Stats were dropped
> {code}
> [vd1309.halxg.cloudera.com:21000] > drop incremental stats 100k_partitions_1m_files_8
partition(part="1000");
> Query: drop incremental stats 100k_partitions_1m_files_8 partition(part="1000")
> [vd1309.halxg.cloudera.com:21000] > drop incremental stats 100k_partitions_1m_files_8
partition(part="1001");
> Query: drop incremental stats 100k_partitions_1m_files_8 partition(part="1001")
> [vd1309.halxg.cloudera.com:21000] > drop incremental stats 100k_partitions_1m_files_8
partition(part="1034");
> Query: drop incremental stats 100k_partitions_1m_files_8 partition(part="1034")
> [vd1309.halxg.cloudera.com:21000] > drop incremental stats 100k_partitions_1m_files_8
partition(part="1040");
> Query: drop incremental stats 100k_partitions_1m_files_8 partition(part="1040")
> [vd1309.halxg.cloudera.com:21000] > drop incremental stats 100k_partitions_1m_files_8
partition(part="1041");
> Query: drop incremental stats 100k_partitions_1m_files_8 partition(part="1041")
> {code}
> Table stats showed that there is no partitions with incremental stats
> {code}
> show table stats 100k_partitions_1m_files_8
> {code}
> Then explain shows that 5 partitions in the table have stats while the rest are missing
stats.
> {code}
> +------------------------------------------------------------------------------------+
> | Explain String                                                                    
|
> +------------------------------------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=3.76GB VCores=1                           
|
> | WARNING: The following tables are missing relevant table and/or column statistics.
|
> | many_files.100k_partitions_1m_files_8                                             
|
> |                                                                                   
|
> | PLAN-ROOT SINK                                                                    
|
> | |                                                                                 
|
> | 03:AGGREGATE [FINALIZE]                                                           
|
> | |  output: count:merge(*)                                                         
|
> | |  hosts=7 per-host-mem=unavailable                                               
|
> | |  tuple-ids=1 row-size=8B cardinality=1                                          
|
> | |                                                                                 
|
> | 02:EXCHANGE [UNPARTITIONED]                                                       
|
> | |  hosts=7 per-host-mem=unavailable                                               
|
> | |  tuple-ids=1 row-size=8B cardinality=1                                          
|
> | |                                                                                 
|
> | 01:AGGREGATE                                                                      
|
> | |  output: count(*)                                                               
|
> | |  hosts=7 per-host-mem=10.00MB                                                   
|
> | |  tuple-ids=1 row-size=8B cardinality=1                                          
|
> | |                                                                                 
|
> | 00:SCAN HDFS [many_files.100k_partitions_1m_files_8, RANDOM]                      
|
> |    partitions=100000/100000 files=928984 size=116.44MB                            
|
> |    table stats: 90 rows total (99995 partition(s) missing stats)                  
|
> |    column stats: all                                                              
|
> |    hosts=7 per-host-mem=3.75GB                                                    
|
> |    tuple-ids=0 row-size=0B cardinality=90                                         
|
> +------------------------------------------------------------------------------------+
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message