impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mostafa Mokhtar (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5086) Planner unconditionally prints missing stats warning when mt_dop is set
Date Fri, 17 Mar 2017 06:27:41 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mostafa Mokhtar resolved IMPALA-5086.
-------------------------------------
    Resolution: Duplicate

Fixed in IMPALA-4981. 
Ran against build pre fix. 

> Planner unconditionally prints missing stats warning when mt_dop is set
> -----------------------------------------------------------------------
>
>                 Key: IMPALA-5086
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5086
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 2.8.0
>            Reporter: Mostafa Mokhtar
>            Assignee: Alexander Behm
>            Priority: Minor
>
> When mt_dop is enabled the Planner reports that one of the tables is missing stats which
isn't correct. 
> {code}
> [foo.com:21000] > set mt_dop=4;
> MT_DOP set to 4
> [foo.com:21000] > explain select l_orderkey, l_partkey,
>        count(*) AS cnt
> FROM lineitem
> GROUP BY l_orderkey,l_partkey
> HAVING count(*) > 9999999999999;
> Query: explain select l_orderkey, l_partkey,
>        count(*) AS cnt
> FROM lineitem
> GROUP BY l_orderkey,l_partkey
> HAVING count(*) > 9999999999999
> +-----------------------------------------------------------------------+
> | Explain String                                                        |
> +-----------------------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=8.64TB VCores=2               |
> | WARNING: Spilling is disabled for this query as a safety guard.       |
> | Reason: Query option disable_unsafe_spills is set, at least one table |
> | is missing relevant stats, and no plan hints were given.              |
> |                                                                       |
> | PLAN-ROOT SINK                                                        |
> | |                                                                     |
> | 04:EXCHANGE [UNPARTITIONED]                                           |
> | |                                                                     |
> | 03:AGGREGATE [FINALIZE]                                               |
> | |  output: count:merge(*)                                             |
> | |  group by: l_orderkey, l_partkey                                    |
> | |  having: count(*) > 9999999999999                                   |
> | |                                                                     |
> | 02:EXCHANGE [HASH(l_orderkey,l_partkey)]                              |
> | |                                                                     |
> | 01:AGGREGATE [STREAMING]                                              |
> | |  output: count(*)                                                   |
> | |  group by: l_orderkey, l_partkey                                    |
> | |                                                                     |
> | 00:SCAN HDFS [tpch_30000_parquet.lineitem]                            |
> |    partitions=2526/2526 files=28976 size=6.89TB                       |
> +-----------------------------------------------------------------------+
> Fetched 22 row(s) in 0.07s
> [foo.com:21000] > set mt_dop=0;
> MT_DOP set to 0
> [foo.com:21000] > explain select l_orderkey, l_partkey,
>        count(*) AS cnt
> FROM lineitem
> GROUP BY l_orderkey,l_partkey
> HAVING count(*) > 9999999999999;
> Query: explain select l_orderkey, l_partkey,
>        count(*) AS cnt
> FROM lineitem
> GROUP BY l_orderkey,l_partkey
> HAVING count(*) > 9999999999999
> +---------------------------------------------------------+
> | Explain String                                          |
> +---------------------------------------------------------+
> | Estimated Per-Host Requirements: Memory=8.64TB VCores=2 |
> |                                                         |
> | PLAN-ROOT SINK                                          |
> | |                                                       |
> | 04:EXCHANGE [UNPARTITIONED]                             |
> | |                                                       |
> | 03:AGGREGATE [FINALIZE]                                 |
> | |  output: count:merge(*)                               |
> | |  group by: l_orderkey, l_partkey                      |
> | |  having: count(*) > 9999999999999                     |
> | |                                                       |
> | 02:EXCHANGE [HASH(l_orderkey,l_partkey)]                |
> | |                                                       |
> | 01:AGGREGATE [STREAMING]                                |
> | |  output: count(*)                                     |
> | |  group by: l_orderkey, l_partkey                      |
> | |                                                       |
> | 00:SCAN HDFS [tpch_30000_parquet.lineitem]              |
> |    partitions=2526/2526 files=28976 size=6.89TB         |
> +---------------------------------------------------------+
> Fetched 19 row(s) in 0.06s
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message