Return-Path: X-Original-To: apmail-drill-dev-archive@www.apache.org Delivered-To: apmail-drill-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B55E110041 for ; Tue, 5 May 2015 20:25:03 +0000 (UTC) Received: (qmail 26651 invoked by uid 500); 5 May 2015 20:24:58 -0000 Delivered-To: apmail-drill-dev-archive@drill.apache.org Received: (qmail 26599 invoked by uid 500); 5 May 2015 20:24:58 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Received: (qmail 26580 invoked by uid 500); 5 May 2015 20:24:58 -0000 Delivered-To: apmail-incubator-drill-dev@incubator.apache.org Received: (qmail 26573 invoked by uid 99); 5 May 2015 20:24:58 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 May 2015 20:24:58 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 785791DBE80; Tue, 5 May 2015 20:24:57 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============2477967234892295100==" MIME-Version: 1.0 Subject: Re: Review Request 33829: DRILL-2757: Verify operators correctly handle low memory conditions and cancellations From: "Steven Phillips" To: "Jacques Nadeau" , "Steven Phillips" , "Hanifi Gunes" Cc: "abdelhakim deneche" , "drill" Date: Tue, 05 May 2015 20:24:57 -0000 Message-ID: <20150505202457.20176.90914@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: "Steven Phillips" X-ReviewGroup: drill-git X-ReviewRequest-URL: https://reviews.apache.org/r/33829/ X-Sender: "Steven Phillips" References: <20150505025155.20176.76056@reviews.apache.org> In-Reply-To: <20150505025155.20176.76056@reviews.apache.org> Reply-To: "Steven Phillips" X-ReviewRequest-Repository: drill-git --===============2477967234892295100== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/33829/#review82574 ----------------------------------------------------------- exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java Does this exception need to be part of the signature? Is it just for documentation purposes? exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java Should we also catch OutOfMemoryRuntimeException? - Steven Phillips On May 5, 2015, 2:51 a.m., abdelhakim deneche wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/33829/ > ----------------------------------------------------------- > > (Updated May 5, 2015, 2:51 a.m.) > > > Review request for drill, Hanifi Gunes, Jacques Nadeau, and Steven Phillips. > > > Bugs: DRILL-2757 > https://issues.apache.org/jira/browse/DRILL-2757 > > > Repository: drill-git > > > Description > ------- > > includes: > [DRILL-2893](https://issues.apache.org/jira/browse/DRILL-2893): ScanBatch throws a NullPointerException instead of returning OUT_OF_MEMORY > [DRILL-2894](https://issues.apache.org/jira/browse/DRILL-2894): FixedValueVectors shouldn't set it's data buffer to null when it fails to allocate it > [DRILL-2895](https://issues.apache.org/jira/browse/DRILL-2895): AbstractRecordBatch.buildSchema() should properly handle OUT_OF_MEMORY outcome > [DRILL-2905](https://issues.apache.org/jira/browse/DRILL-2905): RootExec implementations should properly handle IterOutcome.OUT_OF_MEMORY > [DRILL-2920](https://issues.apache.org/jira/browse/DRILL-2920): properly handle OutOfMemoryException > [DRILL-2947](https://issues.apache.org/jira/browse/DRILL-2947): AllocationHelper.allocateNew() doesn't have a consistent behavior when it can't allocate > > also: > - improved how system errors are displayed > - added UserException.memoryError() with a pre assigned error message > - injection site in ScanBatch and unit test that runs various tpch queries and injects > an exception in the ScanBatch that will cause an OUT_OF_MEMORY outcome to be sent > > > Diffs > ----- > > common/src/main/java/org/apache/drill/common/exceptions/ErrorHelper.java 4da4ee8 > common/src/main/java/org/apache/drill/common/exceptions/UserException.java 9283339 > common/src/test/java/org/apache/drill/common/exceptions/TestUserException.java a145f95 > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 6a924b7 > exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 8a4b663 > exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 09a7568 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 4700dbd > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java 5b4d7bd > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java 67062f3 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java 9f6bea9 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java 15fb7b5 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java b753574 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java 1b90dd8 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java c1c5cb9 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggTemplate.java 86f3100 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java d2282c8 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterTemplate2.java 26f2657 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java dd53477 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java 6466f70 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java d20bfa1 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java eff9e61 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java c36b0d3 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java cf7ba16 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java 35bf3cd > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java 7b9fffb > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java 74b7d85 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java aa9297e > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java af45815 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java d7ea3bb > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java 86d11d5 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java e88bc67 > exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java 4e348bb > exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java 7c77ca2 > exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 2fc5bf3 > exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 4249cbe > exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java ddb828c > exec/java-exec/src/test/java/org/apache/drill/TestOutOfMemoryOutcome.java PRE-CREATION > > Diff: https://reviews.apache.org/r/33829/diff/ > > > Testing > ------- > > added new test class "TestOutOfMemory" that runs several tpch queries and injects an OutOfMemoryException in the ScanBatch. This causes an OUT_OF_MEMORY outcome to be propagated through the execution stack. > > All unit tests are passing, along with functional and tpch100 > > > Thanks, > > abdelhakim deneche > > --===============2477967234892295100==--