impala-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Behm (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (IMPALA-5201) Planner test failures: TpchNested, ConstantFolding, MtDopValidation
Date Wed, 12 Apr 2017 18:13:41 GMT

     [ https://issues.apache.org/jira/browse/IMPALA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Behm resolved IMPALA-5201.
------------------------------------
    Resolution: Duplicate

Duplicate of IMPALA-3887.

Look at the verbose explain output, #hosts=2 but should be 3. This means that at the time
of metadata loading not all replicas were available yet.

> Planner test failures: TpchNested, ConstantFolding, MtDopValidation
> -------------------------------------------------------------------
>
>                 Key: IMPALA-5201
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5201
>             Project: IMPALA
>          Issue Type: Bug
>          Components: Frontend
>    Affects Versions: Impala 2.8.0
>            Reporter: Matthew Jacobs
>            Assignee: Alexander Behm
>            Priority: Critical
>              Labels: broken-build
>         Attachments: planner-test-failure-detail.txt
>
>
> In a recent jenkins run, the following FE tests failed. I don't see any other indications
that anything else went wrong from the console output, but my guess is that tpch_nested_parquet
didn't have stats for some reason.
> {code}
> 23:31:10 testTpchNested(org.apache.impala.planner.PlannerTest)  Time elapsed: 1.905 sec
 <<< FAILURE!
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:741)
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:750)
> 23:31:10 	at org.apache.impala.planner.PlannerTest.testTpchNested(PlannerTest.java:216)
> 23:31:10 testConstantFolding(org.apache.impala.planner.PlannerTest)  Time elapsed: 1.82
sec  <<< FAILURE!
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:741)
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:705)
> 23:31:10 	at org.apache.impala.planner.PlannerTest.testConstantFolding(PlannerTest.java:56)
> 23:31:10 testMtDopValidation(org.apache.impala.planner.PlannerTest)  Time elapsed: 0.213
sec  <<< FAILURE!
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:741)
> 23:31:10 	at org.apache.impala.planner.PlannerTestBase.runPlannerTestFile(PlannerTestBase.java:705)
> 23:31:10 	at org.apache.impala.planner.PlannerTest.testMtDopValidation(PlannerTest.java:323)
> {code}
> Attached is the output from the failed tests. Here's a single example:
> {code}
> REGRESSION:  org.apache.impala.planner.PlannerTest.testTpchNested
> Error Message:
> Section DISTRIBUTEDPLAN of query:
> select
>   sum(l_extendedprice) / 7.0 as avg_yearly
> from
>   customer.c_orders.o_lineitems l,
>   part p
> where
>   p_partkey = l_partkey
>   and p_brand = 'Brand#23'
>   and p_container = 'MED BOX'
>   and l_quantity < (
>     select
>       0.2 * avg(l_quantity)
>     from
>       customer.c_orders.o_lineitems l
>     where
>       l_partkey = p_partkey
>   )
> Actual does not match expected result:
> PLAN-ROOT SINK
> |
> 12:AGGREGATE [FINALIZE]
> |  output: sum:merge(l_extendedprice)
> |
> 11:EXCHANGE [UNPARTITIONED]
> |
> 06:AGGREGATE
> |  output: sum(l_extendedprice)
> |
> 05:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> |  hash predicates: p_partkey = l_partkey
> |  other join predicates: l_quantity < 0.2 * avg(l_quantity)
> |  runtime filters: RF000 <- l_partkey
> |
> |--10:EXCHANGE [BROADCAST]
> |  |
> |  09:AGGREGATE [FINALIZE]
> |  |  output: avg:merge(l_quantity)
> |  |  group by: l_partkey
> |  |
> |  08:EXCHANGE [HASH(l_partkey)]
> |  |
> |  03:AGGREGATE [STREAMING]
> |  |  output: avg(l_quantity)
> |  |  group by: l_partkey
> |  |
> |  02:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l]
> |     partitions=1/1 files=4 size=292.36MB
> |
> 04:HASH JOIN [INNER JOIN, BROADCAST]
> |  hash predicates: l_partkey = p_partkey
> |  runtime filters: RF001 <- p_partkey
> |
> |--07:EXCHANGE [BROADCAST]
> |  |
> |  01:SCAN HDFS [tpch_nested_parquet.part p]
> |     partitions=1/1 files=1 size=6.23MB
> |     predicates: p_container = 'MED BOX', p_brand = 'Brand#23'
> |     runtime filters: RF000 -> p_partkey
> |
> 00:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l]
>    partitions=1/1 files=4 size=292.36MB
>    runtime filters: RF000 -> l.l_partkey, RF001 -> l_partkey
> Expected:
> PLAN-ROOT SINK
> |
> 12:AGGREGATE [FINALIZE]
> |  output: sum:merge(l_extendedprice)
> |
> 11:EXCHANGE [UNPARTITIONED]
> |
> 06:AGGREGATE
> |  output: sum(l_extendedprice)
> |
> 05:HASH JOIN [LEFT SEMI JOIN, PARTITIONED]
> |  hash predicates: p_partkey = l_partkey
> |  other join predicates: l_quantity < 0.2 * avg(l_quantity)
> |  runtime filters: RF000 <- l_partkey
> |
> |--09:AGGREGATE [FINALIZE]
> |  |  output: avg:merge(l_quantity)
> |  |  group by: l_partkey
> |  |
> |  08:EXCHANGE [HASH(l_partkey)]
> |  |
> |  03:AGGREGATE [STREAMING]
> |  |  output: avg(l_quantity)
> |  |  group by: l_partkey
> |  |
> |  02:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l]
> |     partitions=1/1 files=4 size=577.87MB
> |
> 10:EXCHANGE [HASH(p_partkey)]
> |
> 04:HASH JOIN [INNER JOIN, BROADCAST]
> |  hash predicates: l_partkey = p_partkey
> |  runtime filters: RF001 <- p_partkey
> |
> |--07:EXCHANGE [BROADCAST]
> |  |
> |  01:SCAN HDFS [tpch_nested_parquet.part p]
> |     partitions=1/1 files=1 size=6.20MB
> |     predicates: p_container = 'MED BOX', p_brand = 'Brand#23'
> |     runtime filters: RF000 -> p_partkey
> |
> 00:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l]
>    partitions=1/1 files=4 size=577.87MB
>    runtime filters: RF000 -> l.l_partkey, RF001 -> l_partkey
> Verbose plan:
> F04:PLAN FRAGMENT [UNPARTITIONED]
>   PLAN-ROOT SINK
>   |
>   12:AGGREGATE [FINALIZE]
>   |  output: sum:merge(l_extendedprice)
>   |  hosts=2 per-host-mem=unavailable
>   |  tuple-ids=6 row-size=16B cardinality=1
>   |
>   11:EXCHANGE [UNPARTITIONED]
>      hosts=2 per-host-mem=unavailable
>      tuple-ids=6 row-size=16B cardinality=1
> F00:PLAN FRAGMENT [RANDOM]
>   DATASTREAM SINK [FRAGMENT=F04, EXCHANGE=11, UNPARTITIONED]
>   06:AGGREGATE
>   |  output: sum(l_extendedprice)
>   |  hosts=2 per-host-mem=10.00MB
>   |  tuple-ids=6 row-size=16B cardinality=1
>   |
>   05:HASH JOIN [LEFT SEMI JOIN, BROADCAST]
>   |  hash predicates: p_partkey = l_partkey
>   |  other join predicates: l_quantity < 0.2 * avg(l_quantity)
>   |  runtime filters: RF000 <- l_partkey
>   |  hosts=2 per-host-mem=251.77MB
>   |  tuple-ids=0,1 row-size=80B cardinality=15000000
>   |
>   |--10:EXCHANGE [BROADCAST]
>   |     hosts=2 per-host-mem=0B
>   |     tuple-ids=4 row-size=16B cardinality=15000000
>   |
>   04:HASH JOIN [INNER JOIN, BROADCAST]
>   |  hash predicates: l_partkey = p_partkey
>   |  runtime filters: RF001 <- p_partkey
>   |  hosts=2 per-host-mem=59.70KB
>   |  tuple-ids=0,1 row-size=80B cardinality=15000000
>   |
>   |--07:EXCHANGE [BROADCAST]
>   |     hosts=1 per-host-mem=0B
>   |     tuple-ids=1 row-size=56B cardinality=1000
>   |
>   00:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l, RANDOM]
>      partitions=1/1 files=4 size=292.36MB
>      runtime filters: RF000 -> l.l_partkey, RF001 -> l_partkey
>      table stats: 150000 rows total
>      column stats: all
>      hosts=2 per-host-mem=264.00MB
>      tuple-ids=0 row-size=24B cardinality=15000000
> F01:PLAN FRAGMENT [RANDOM]
>   DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=07, BROADCAST]
>   01:SCAN HDFS [tpch_nested_parquet.part p, RANDOM]
>      partitions=1/1 files=1 size=6.23MB
>      predicates: p_container = 'MED BOX', p_brand = 'Brand#23'
>      runtime filters: RF000 -> p_partkey
>      table stats: 200000 rows total
>      column stats: all
>      parquet statistics predicates: p_container = 'MED BOX', p_brand = 'Brand#23'
>      parquet dictionary predicates: p_container = 'MED BOX', p_brand = 'Brand#23'
>      hosts=1 per-host-mem=48.00MB
>      tuple-ids=1 row-size=56B cardinality=1000
> F03:PLAN FRAGMENT [HASH(l_partkey)]
>   DATASTREAM SINK [FRAGMENT=F00, EXCHANGE=10, BROADCAST]
>   09:AGGREGATE [FINALIZE]
>   |  output: avg:merge(l_quantity)
>   |  group by: l_partkey
>   |  hosts=2 per-host-mem=128.00MB
>   |  tuple-ids=4 row-size=16B cardinality=15000000
>   |
>   08:EXCHANGE [HASH(l_partkey)]
>      hosts=2 per-host-mem=0B
>      tuple-ids=3 row-size=16B cardinality=15000000
> F02:PLAN FRAGMENT [RANDOM]
>   DATASTREAM SINK [FRAGMENT=F03, EXCHANGE=08, HASH(l_partkey)]
>   03:AGGREGATE [STREAMING]
>   |  output: avg(l_quantity)
>   |  group by: l_partkey
>   |  hosts=2 per-host-mem=128.00MB
>   |  tuple-ids=3 row-size=16B cardinality=15000000
>   |
>   02:SCAN HDFS [tpch_nested_parquet.customer.c_orders.o_lineitems l, RANDOM]
>      partitions=1/1 files=4 size=292.36MB
>      table stats: 150000 rows total
>      column stats: all
>      hosts=2 per-host-mem=176.00MB
>      tuple-ids=2 row-size=16B cardinality=15000000
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message