Mailing-List: contact issues-help@impala.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@impala.incubator.apache.org
Date: Wed, 26 Jul 2017 22:44:01 +0000 (UTC)
From: "Tim Armstrong (JIRA)" <jira@apache.org>
To: issues@impala.incubator.apache.org
Message-ID: <JIRA.13088990.1500624771000.22334.1501109041446@Atlassian.JIRA>
In-Reply-To: <JIRA.13088990.1500624771000@Atlassian.JIRA>
References: <JIRA.13088990.1500624771000@Atlassian.JIRA> <JIRA.13088990.1500624771397@jira-lw-us.apache.org>
Subject: [jira] [Resolved] (IMPALA-5691) test_low_mem_limit_q18 is flaky
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 26 Jul 2017 22:44:09 -0000


     [ https://issues.apache.org/jira/browse/IMPALA-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-5691.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: Impala 2.10.0

Will close for now and reopen if we see it again.

IMPALA-5691: recalibrate mem limit for Q18

I ran the stress test binary search locally and it produced a slightly
higher number for Q18 than the hardcoded value. This is enough to move
it above one of the thresholds, so may reduce flakiness.

Testing:
I wasn't able to reproduce the flakiness locally, so can't confirm
this fixes it.

Change-Id: I1ffa969061a52730c5147d142dcd2e3cb3626590
Reviewed-on: http://gerrit.cloudera.org:8080/7512
Reviewed-by: Matthew Jacobs <mj@cloudera.com>
Tested-by: Impala Public Jenkins
---

> test_low_mem_limit_q18 is flaky
> -------------------------------
>
>                 Key: IMPALA-5691
>                 URL: https://issues.apache.org/jira/browse/IMPALA-5691
>             Project: IMPALA
>          Issue Type: Bug
>    Affects Versions: Impala 2.9.0
>            Reporter: Anuj Phadke
>            Assignee: Tim Armstrong
>            Priority: Critical
>              Labels: flaky-test
>             Fix For: Impala 2.10.0
>
>
> test_low_mem_limit_q18 failed while running this job -
> https://jenkins.impala.io/job/gerrit-verify-dryrun/900/console
> This does not consistently happen though.
> 22:55:47 ]  TestTpchMemLimitError.test_low_mem_limit_q18[mem_limit: 275 | exec_option: {'batch_size': 0, 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': False, 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format: parquet/none] 
> 22:55:47 ] [gw1] linux2 -- Python 2.7.6 /home/ubuntu/Impala/bin/../infra/python/env/bin/python
> 22:55:47 ] query_test/test_mem_usage_scaling.py:187: in test_low_mem_limit_q18
> 22:55:47 ]     self.low_memory_limit_test(vector, 'tpch-q18', self.MIN_MEM_FOR_TPCH['Q18'])
> 22:55:47 ] query_test/test_mem_usage_scaling.py:101: in low_memory_limit_test
> 22:55:47 ]     assert TestLowMemoryLimits.EXPECTED_ERROR_MSG in str(e)
> 22:55:47 ] E   assert 'Memory limit exceeded' in 'ImpalaBeeswaxException:\n Query aborted:Cannot perform aggregation at node with id 14. Repartitioning did not reduce the size of a spilled partition. Repartitioning level 6. Number of rows 1.\n\n'
> 22:55:47 ] E    +  where 'Memory limit exceeded' = TestLowMemoryLimits.EXPECTED_ERROR_MSG
> 22:55:47 ] E    +  and   'ImpalaBeeswaxException:\n Query aborted:Cannot perform aggregation at node with id 14. Repartitioning did not reduce the size of a spilled partition. Repartitioning level 6. Number of rows 1.\n\n' = str(ImpalaBeeswaxException())
> 22:55:47 ] ----------------------------- Captured stderr call -----------------------------
> 22:55:47 ] -- executing against localhost:21000
> 22:55:47 ] use tpch_parquet;
> 22:55:47 ] 
> 22:55:47 ] SET batch_size=0;
> 22:55:47 ] SET num_nodes=0;
> 22:55:47 ] SET disable_codegen_rows_threshold=0;
> 22:55:47 ] SET disable_codegen=False;
> 22:55:47 ] SET abort_on_error=1;
> 22:55:47 ] SET mem_limit=275m;
> 22:55:47 ] SET exec_single_node_rows_threshold=0;
> 22:55:47 ] MainThread: Query Name: 
> 22:55:47 ] TPCH-Q18
> 22:55:47 ] 
> 22:55:47 ] -- executing against localhost:21000
> 22:55:47 ] select
> 22:55:47 ]   c_name,
> 22:55:47 ]   c_custkey,
> 22:55:47 ]   o_orderkey,
> 22:55:47 ]   o_orderdate,
> 22:55:47 ]   o_totalprice,
> 22:55:47 ]   sum(l_quantity)
> 22:55:47 ] from
> 22:55:47 ]   customer,
> 22:55:47 ]   orders,
> 22:55:47 ]   lineitem
> 22:55:47 ] where
> 22:55:47 ]   o_orderkey in (
> 22:55:47 ]     select
> 22:55:47 ]       l_orderkey
> 22:55:47 ]     from
> 22:55:47 ]       lineitem
> 22:55:47 ]     group by
> 22:55:47 ]       l_orderkey
> 22:55:47 ]     having
> 22:55:47 ]       sum(l_quantity) > 300
> 22:55:47 ]     )
> 22:55:47 ]   and c_custkey = o_custkey
> 22:55:47 ]   and o_orderkey = l_orderkey
> 22:55:47 ] group by
> 22:55:47 ]   c_name,
> 22:55:47 ]   c_custkey,
> 22:55:47 ]   o_orderkey,
> 22:55:47 ]   o_orderdate,
> 22:55:47 ]   o_totalprice
> 22:55:47 ] order by
> 22:55:47 ]   o_totalprice desc,
> 22:55:47 ]   o_orderdate
> 22:55:47 ] limit 100;
> 22:55:47 ] 
> 22:55:47 ] ======= 1 failed, 1436 passed, 42 skipped, 36 xfailed in 1806.79 seconds =======


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)