Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8F340200D2F for ; Wed, 1 Nov 2017 07:44:56 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8DBF4160BEA; Wed, 1 Nov 2017 06:44:56 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D3AB9160BE6 for ; Wed, 1 Nov 2017 07:44:55 +0100 (CET) Received: (qmail 8700 invoked by uid 500); 1 Nov 2017 06:44:54 -0000 Mailing-List: contact reviews-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list reviews@impala.incubator.apache.org Received: (qmail 8689 invoked by uid 99); 1 Nov 2017 06:44:54 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Nov 2017 06:44:54 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 8C483D47F0 for ; Wed, 1 Nov 2017 06:44:53 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.362 X-Spam-Level: ** X-Spam-Status: No, score=2.362 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=2, RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id B8zfR5QEUPmD for ; Wed, 1 Nov 2017 06:44:52 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 924685F121 for ; Wed, 1 Nov 2017 06:44:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id vA16ipkF009690; Wed, 1 Nov 2017 06:44:51 GMT X-Gerrit-PatchSet: 1 Date: Wed, 1 Nov 2017 06:44:51 +0000 From: "Tim Armstrong (Code Review)" To: Marcel Kornacker , impala-cr@cloudera.com, reviews@impala.incubator.apache.org Message-ID: X-Gerrit-MessageType: newchange Subject: =?UTF-8?Q?=5BImpala-ASF-CR=5D_IMPALA-6137=3A_fix_text_scanner_split_delim_mem_mgmt=0A?= X-Gerrit-Change-Id: Iddbb5cf6acc8f0b0e0b4c205c334f21e03d06f1c X-Gerrit-Change-Number: 8438 X-Gerrit-ChangeURL: X-Gerrit-Commit: c281e5f27d48126d1962b3cc6f90469db02337bd Reply-To: tarmstrong@cloudera.com, impala-cr@cloudera.com, marcelk@gmail.com, reviews@impala.incubator.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.14.2 Content-Type: multipart/alternative; boundary="33ifFMQql58="; charset=UTF-8 archived-at: Wed, 01 Nov 2017 06:44:56 -0000 --33ifFMQql58= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Tim Armstrong has uploaded this change for review=2E ( http://gerrit=2Eclou= dera=2Eorg:8080/8438 Change subject: IMPALA-6137: fix text scanner split = delim mem mgmt =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= IMPALA-6137: fix text scanner split delim mem mgmt The bug was that the = buffer pointed to by byte_buffer_ could be freed by ReleaseCompletedResourc= es() before CheckForSplitDelimiter() was called=2E The simple fix is to co= py out the single byte that is needed each time the buffer is filled=2E Te= sting: Ran exhaustive query tests under ASAN with --disable_mem_pools=3Dtru= e=2E Before the change test_text_split_delimiters reliably caused an ASAN = failure when run with --disable_mem_pools=3Dtrue=2E We should get this cove= rage automatically once the I/O mgr switches to using the buffer pool, whic= h uses ASAN poisoning on freed buffers=2E Change-Id: Iddbb5cf6acc8f0b0e0b4= c205c334f21e03d06f1c --- M be/src/exec/hdfs-text-scanner=2Ecc M be/src/exec= /hdfs-text-scanner=2Eh 2 files changed, 10 insertions(+), 1 deletion(-) = git pull ssh://gerrit=2Ecloudera=2Eorg:29418/Impala-ASF refs/changes/38/8= 438/1 -- To view, visit http://gerrit=2Ecloudera=2Eorg:8080/8438 To unsubs= cribe, visit http://gerrit=2Ecloudera=2Eorg:8080/settings Gerrit-Project: = Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Chang= e-Id: Iddbb5cf6acc8f0b0e0b4c205c334f21e03d06f1c Gerrit-Change-Number: 8438 = Gerrit-PatchSet: 1 Gerrit-Owner: Tim Armstrong --33ifFMQql58=--