Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id DF428200D0A for ; Tue, 29 Aug 2017 18:00:11 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id DD3C3166F42; Tue, 29 Aug 2017 16:00:11 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2CAE0166F43 for ; Tue, 29 Aug 2017 18:00:11 +0200 (CEST) Received: (qmail 79110 invoked by uid 500); 29 Aug 2017 16:00:08 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 78554 invoked by uid 99); 29 Aug 2017 16:00:07 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Aug 2017 16:00:07 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 6E7731A0A8D for ; Tue, 29 Aug 2017 15:48:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.362 X-Spam-Level: X-Spam-Status: No, score=0.362 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id bqL3EY0OBQJu for ; Tue, 29 Aug 2017 15:48:18 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id A25E95F6D3 for ; Tue, 29 Aug 2017 15:48:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id v7TFmHEA016056; Tue, 29 Aug 2017 15:48:17 GMT Message-Id: <201708291548.v7TFmHEA016056@ip-10-146-233-104.ec2.internal> Date: Tue, 29 Aug 2017 15:48:17 +0000 From: "Tim Armstrong (Code Review)" To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org Reply-To: tarmstrong@cloudera.com X-Gerrit-MessageType: newpatchset Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5855=3A_reserve_enough_memory_for_preaggs=0A?= X-Gerrit-Change-Id: I870fbe2f1da01c6123d3716a1198376f9a454c3b X-Gerrit-ChangeURL: X-Gerrit-Commit: 6a4254acb5f1ba164a9b2fff8b610818e472ef55 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.7 archived-at: Tue, 29 Aug 2017 16:00:12 -0000 Tim Armstrong has uploaded a new patch set (#2). Change subject: IMPALA-5855: reserve enough memory for preaggs ...................................................................... IMPALA-5855: reserve enough memory for preaggs The calculation in the planner failed to account for the behaviour of Suballocator, which needs to obtain at least one buffer to allocate any memory. Testing: Added a regression test that caused a crash before the fix. Updated planner tests. Was able to run local stress test binary search to completion (it previously crashed). Change-Id: I870fbe2f1da01c6123d3716a1198376f9a454c3b --- M fe/src/main/java/org/apache/impala/planner/AggregationNode.java M testdata/workloads/functional-planner/queries/PlannerTest/max-row-size.test M testdata/workloads/functional-planner/queries/PlannerTest/mt-dop-validation.test M testdata/workloads/functional-planner/queries/PlannerTest/partition-pruning.test M testdata/workloads/functional-planner/queries/PlannerTest/resource-requirements.test M testdata/workloads/functional-planner/queries/PlannerTest/spillable-buffer-sizing.test M testdata/workloads/functional-query/queries/QueryTest/aggregation.test 7 files changed, 85 insertions(+), 71 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/7871/2 -- To view, visit http://gerrit.cloudera.org:8080/7871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I870fbe2f1da01c6123d3716a1198376f9a454c3b Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong