Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 148182007D1 for ; Thu, 12 May 2016 20:03:44 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 13364160939; Thu, 12 May 2016 18:03:44 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 5BC381602BF for ; Thu, 12 May 2016 20:03:43 +0200 (CEST) Received: (qmail 65567 invoked by uid 500); 12 May 2016 18:03:42 -0000 Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@impala.incubator.apache.org Delivered-To: mailing list dev@impala.incubator.apache.org Received: (qmail 65556 invoked by uid 99); 12 May 2016 18:03:42 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 12 May 2016 18:03:42 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 1F68F1A1162 for ; Thu, 12 May 2016 18:03:42 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.362 X-Spam-Level: X-Spam-Status: No, score=0.362 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id YJT_mlnGL3_t for ; Thu, 12 May 2016 18:03:40 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 9241E5F484 for ; Thu, 12 May 2016 18:03:39 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id u4CI3d7I024568; Thu, 12 May 2016 18:03:39 GMT Date: Thu, 12 May 2016 18:03:38 +0000 From: "Tim Armstrong (Code Review)" To: impala-cr@cloudera.com, dev@impala.incubator.apache.org Message-ID: Reply-To: tarmstrong@cloudera.com X-Gerrit-MessageType: newchange Subject: =?UTF-8?Q?[Impala-CR](cdh5-trunk)_PREVIEW:_Use_mmap_instead_of_malloc_for_buffer_pool=0A?= X-Gerrit-Change-Id: Ifbc748f74adcbbdcfa45f3ec7df98284925acbd6 X-Gerrit-ChangeURL: X-Gerrit-Commit: ccbe101effcf87872398059b5aca5e6b7765fc2c MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.10-rc0 archived-at: Thu, 12 May 2016 18:03:44 -0000 Tim Armstrong has uploaded a new change for review. http://gerrit.cloudera.org:8080/3046 Change subject: PREVIEW: Use mmap instead of malloc for buffer pool ...................................................................... PREVIEW: Use mmap instead of malloc for buffer pool Allocate with mmap instead of TCMalloc to give more control over memory usage. Also allocate huge pages when possible to reduce TLB pressure. We also need some additional memory metrics, since we previously relied on the assumption that all memory was allocated through TCMalloc. Change-Id: Ifbc748f74adcbbdcfa45f3ec7df98284925acbd6 --- M be/src/bufferpool/CMakeLists.txt A be/src/bufferpool/buffer-allocator-test.cc M be/src/bufferpool/buffer-allocator.cc M be/src/bufferpool/buffer-allocator.h M be/src/bufferpool/buffer-pool.h M be/src/catalog/catalogd-main.cc M be/src/runtime/exec-env.cc M be/src/statestore/statestored-main.cc M be/src/util/memory-metrics.cc M be/src/util/memory-metrics.h M be/src/util/metrics-test.cc M be/src/util/metrics.h M common/thrift/generate_error_codes.py M common/thrift/metrics.json 14 files changed, 335 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/46/3046/1 -- To view, visit http://gerrit.cloudera.org:8080/3046 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifbc748f74adcbbdcfa45f3ec7df98284925acbd6 Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Tim Armstrong