Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C4610200C72 for ; Fri, 12 May 2017 17:36:25 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C2D97160BB8; Fri, 12 May 2017 15:36:25 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1535D160BA3 for ; Fri, 12 May 2017 17:36:24 +0200 (CEST) Received: (qmail 42779 invoked by uid 500); 12 May 2017 15:36:24 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 42767 invoked by uid 99); 12 May 2017 15:36:24 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 12 May 2017 15:36:24 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id AF027C1A63 for ; Fri, 12 May 2017 15:36:23 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.363 X-Spam-Level: X-Spam-Status: No, score=0.363 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id k7A4zIsRiJUS for ; Fri, 12 May 2017 15:36:23 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 7EF195FB40 for ; Fri, 12 May 2017 15:36:22 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id v4CFaFU5026776; Fri, 12 May 2017 15:36:15 GMT Date: Fri, 12 May 2017 15:36:15 +0000 From: "Tim Armstrong (Code Review)" To: impala-cr@cloudera.com, reviews@impala.incubator.apache.org Message-ID: Reply-To: tarmstrong@cloudera.com X-Gerrit-MessageType: newchange Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-5304=3A_reduce_transfer_of_Parquet_decompression_buffers=0A?= X-Gerrit-Change-Id: I2dbd749f43078b222ff8e1ddcec840986c466de6 X-Gerrit-ChangeURL: X-Gerrit-Commit: dd91a2a1728c225754bdc4845e69b1f7cedf356f MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.7 archived-at: Fri, 12 May 2017 15:36:26 -0000 Tim Armstrong has uploaded a new change for review. http://gerrit.cloudera.org:8080/6876 Change subject: IMPALA-5304: reduce transfer of Parquet decompression buffers ...................................................................... IMPALA-5304: reduce transfer of Parquet decompression buffers The buffers contain the Parquet DataPages, which need to be attached to the row batch if the rows point to var-len data stored directly in the page. Otherwise the buffers can be discarded once the values in the page have been materialized. This reduces the amount of memory transferred between threads, which is a known TCMalloc anti-pattern. It also allows us to free memory earlier, which may help reduce memory consumption slightly. Testing Ran exhaustive build. Most of the Parquet tests use compressed Parquet, which should exercise this code path. Change-Id: I2dbd749f43078b222ff8e1ddcec840986c466de6 --- M be/src/exec/parquet-column-readers.cc M be/src/exec/parquet-column-readers.h 2 files changed, 18 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/76/6876/1 -- To view, visit http://gerrit.cloudera.org:8080/6876 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2dbd749f43078b222ff8e1ddcec840986c466de6 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong