Return-Path: X-Original-To: apmail-impala-dev-archive@minotaur.apache.org Delivered-To: apmail-impala-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E3A191844C for ; Wed, 6 Apr 2016 23:38:17 +0000 (UTC) Received: (qmail 55247 invoked by uid 500); 6 Apr 2016 23:38:17 -0000 Delivered-To: apmail-impala-dev-archive@impala.apache.org Received: (qmail 55205 invoked by uid 500); 6 Apr 2016 23:38:17 -0000 Mailing-List: contact dev-help@impala.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@impala.incubator.apache.org Delivered-To: mailing list dev@impala.incubator.apache.org Received: (qmail 55194 invoked by uid 99); 6 Apr 2016 23:38:17 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 06 Apr 2016 23:38:17 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id E89A4C0218 for ; Wed, 6 Apr 2016 23:38:16 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.362 X-Spam-Level: X-Spam-Status: No, score=0.362 tagged_above=-999 required=6.31 tests=[RDNS_DYNAMIC=0.363, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id R-WbzPidPzWY for ; Wed, 6 Apr 2016 23:38:14 +0000 (UTC) Received: from ip-10-146-233-104.ec2.internal (ec2-75-101-130-251.compute-1.amazonaws.com [75.101.130.251]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 2CE705F202 for ; Wed, 6 Apr 2016 23:38:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by ip-10-146-233-104.ec2.internal (8.14.4/8.14.4) with ESMTP id u36NcCAP008285; Wed, 6 Apr 2016 23:38:12 GMT Message-Id: <201604062338.u36NcCAP008285@ip-10-146-233-104.ec2.internal> Date: Wed, 6 Apr 2016 23:38:12 +0000 From: "Internal Jenkins (Code Review)" To: Michael Ho , impala-cr@cloudera.com, dev@impala.incubator.apache.org X-Gerrit-MessageType: merged Subject: =?UTF-8?Q?[Impala-CR](cdh5-trunk)_IMPALA-2399:_Memory_limit_checks_for_all_scanners.=0A?= X-Gerrit-Change-Id: I3e5a56501967a58513888917db5ce66dec4fb5ce X-Gerrit-ChangeURL: X-Gerrit-Commit: 3d88b32b1a9890bff8262ee2a48178b8c897e7e9 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.10-rc0 Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-2399: Memory limit checks for all scanners. ...................................................................... IMPALA-2399: Memory limit checks for all scanners. This change replaces all instances of MemPool::Allocate() in avro, text, hbase scanners with MemPool::TryAllocate(). HdfsAvroScanner::MaterializeTuple() has been converted to return a boolean in case of memory allocation failure. The codegen'ed version of MaterializeTuple() will also return a boolean. In the future, we should consider returning Status directly but that will be more involved and best left as a separate change. Change-Id: I3e5a56501967a58513888917db5ce66dec4fb5ce Reviewed-on: http://gerrit.cloudera.org:8080/2568 Reviewed-by: Michael Ho Tested-by: Internal Jenkins --- M be/src/exec/base-sequence-scanner.cc M be/src/exec/data-source-scan-node.cc M be/src/exec/hbase-table-scanner.cc M be/src/exec/hbase-table-scanner.h M be/src/exec/hdfs-avro-scanner-ir.cc M be/src/exec/hdfs-avro-scanner.cc M be/src/exec/hdfs-avro-scanner.h M be/src/exec/hdfs-parquet-scanner.cc M be/src/exec/hdfs-scanner.cc M be/src/exec/hdfs-sequence-scanner.cc M be/src/exec/hdfs-text-scanner.cc M be/src/exec/hdfs-text-scanner.h M be/src/exec/scanner-context.cc M be/src/exec/text-converter.h M be/src/exec/text-converter.inline.h M be/src/runtime/string-buffer.h 16 files changed, 285 insertions(+), 128 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/2568 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3e5a56501967a58513888917db5ce66dec4fb5ce Gerrit-PatchSet: 5 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Michael Ho Gerrit-Reviewer: Dan Hecht Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho