Return-Path: X-Original-To: apmail-drill-dev-archive@www.apache.org Delivered-To: apmail-drill-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0720918F3A for ; Mon, 30 Nov 2015 05:32:40 +0000 (UTC) Received: (qmail 52886 invoked by uid 500); 30 Nov 2015 05:32:39 -0000 Delivered-To: apmail-drill-dev-archive@drill.apache.org Received: (qmail 52828 invoked by uid 500); 30 Nov 2015 05:32:39 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Received: (qmail 52817 invoked by uid 99); 30 Nov 2015 05:32:39 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Nov 2015 05:32:39 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 66FC9DFC8E; Mon, 30 Nov 2015 05:32:39 +0000 (UTC) From: jacques-n To: dev@drill.apache.org Reply-To: dev@drill.apache.org Message-ID: Subject: [GitHub] drill pull request: DRILL-4142: Query/Node memory maximums Content-Type: text/plain Date: Mon, 30 Nov 2015 05:32:39 +0000 (UTC) GitHub user jacques-n opened a pull request: https://github.com/apache/drill/pull/284 DRILL-4142: Query/Node memory maximums Simple query allocator design. Can be enhanced via better workload management flow. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jacques-n/drill DRILL-4142 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/284.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #284 ---- commit 04da30cfd0c37f6289b4ef5a7517c43828845ef5 Author: Chris Westin Date: 2015-11-11T22:57:16Z DRILL-4144: Clean up close behaviors. commit 881284281d1f22db994cde42b876a57882377144 Author: Chris Westin Date: 2015-11-11T22:57:47Z DRILL-4144: Add new allocator commit 14f5b4dbb94a18b9c697560f37e2d2cb666d5f30 Author: Jacques Nadeau Date: 2015-11-16T01:26:02Z DRILL-4144: Allocator Improvements - make Allocator mostly lockless - change BaseAllocator maps to direct references - add documentation around memory management model - move transfer and ownership methods to DrillBuf commit 177cc9ed13d4c28e747caed1f7686ac759e68b1f Author: Jacques Nadeau Date: 2015-11-27T18:26:07Z Improve debug messaging. Fix issue where we were under reference-counting Update accountant to avoid reservation limit failure incorrect accounting. commit 736a10d2a37903f7afcae23a962a94dac6e5f377 Author: Jacques Nadeau Date: 2015-11-27T18:52:08Z Fix/revert sort changes commit 265cbc55117c8bd98497ce5a5fdc0b46926a8d37 Author: Jacques Nadeau Date: 2015-11-27T18:54:31Z Update DrillBuf check to avoid generating verbose string unless required. commit 160cd963ce07b2633f93f07e39937f0a3c0fac55 Author: Jacques Nadeau Date: 2015-11-30T05:29:50Z DRILL-4142: Implement basic query level memory maximums. - Add new AllocatorDecorator helper class. - Update BaseAllocator/BufferAllocator to support unwrap(Class) functionality - Create a new QueryAllocator that survives as long as at least one fragment allocator exists. - Expose synchronized QueryAllocator creation per node via the WorkEventBus - Expose a new SYSTEM-only option: "exec.memory.max.per_query_per_node.mb" with a default of 1TB. ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastructure@apache.org or file a JIRA ticket with INFRA. ---