drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau" <jacques.dr...@gmail.com>
Subject Re: Review Request 33829: DRILL-2757: Verify operators correctly handle low memory conditions and cancellations
Date Tue, 05 May 2015 23:19:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33829/#review82606
-----------------------------------------------------------


Let's just have one exception OutOfMemoryException and make it a RuntimeException.  I don't
know what the purpose of having two different ones is.

- Jacques Nadeau


On May 5, 2015, 2:51 a.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33829/
> -----------------------------------------------------------
> 
> (Updated May 5, 2015, 2:51 a.m.)
> 
> 
> Review request for drill, Hanifi Gunes, Jacques Nadeau, and Steven Phillips.
> 
> 
> Bugs: DRILL-2757
>     https://issues.apache.org/jira/browse/DRILL-2757
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> includes:
> [DRILL-2893](https://issues.apache.org/jira/browse/DRILL-2893): ScanBatch throws a NullPointerException
instead of returning OUT_OF_MEMORY
> [DRILL-2894](https://issues.apache.org/jira/browse/DRILL-2894): FixedValueVectors shouldn't
set it's data buffer to null when it fails to allocate it
> [DRILL-2895](https://issues.apache.org/jira/browse/DRILL-2895): AbstractRecordBatch.buildSchema()
should properly handle OUT_OF_MEMORY outcome
> [DRILL-2905](https://issues.apache.org/jira/browse/DRILL-2905): RootExec implementations
should properly handle IterOutcome.OUT_OF_MEMORY
> [DRILL-2920](https://issues.apache.org/jira/browse/DRILL-2920): properly handle OutOfMemoryException
> [DRILL-2947](https://issues.apache.org/jira/browse/DRILL-2947): AllocationHelper.allocateNew()
doesn't have a consistent behavior when it can't allocate
> 
> also:
> - improved how system errors are displayed
> - added UserException.memoryError() with a pre assigned error message
> - injection site in ScanBatch and unit test that runs various tpch queries and injects
>   an exception in the ScanBatch that will cause an OUT_OF_MEMORY outcome to be sent
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/ErrorHelper.java 4da4ee8 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java 9283339

>   common/src/test/java/org/apache/drill/common/exceptions/TestUserException.java a145f95

>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 6a924b7 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 8a4b663 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 09a7568

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 4700dbd

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
5b4d7bd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
67062f3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
9f6bea9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
15fb7b5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
b753574 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
1b90dd8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
c1c5cb9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggTemplate.java
86f3100 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java
d2282c8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterTemplate2.java
26f2657 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
dd53477 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
6466f70 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java
d20bfa1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java
eff9e61 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
c36b0d3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
cf7ba16 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
35bf3cd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
7b9fffb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
74b7d85 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java
aa9297e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java
af45815 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
d7ea3bb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java
86d11d5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
e88bc67 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
4e348bb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java 7c77ca2

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 2fc5bf3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 4249cbe

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
ddb828c 
>   exec/java-exec/src/test/java/org/apache/drill/TestOutOfMemoryOutcome.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/33829/diff/
> 
> 
> Testing
> -------
> 
> added new test class "TestOutOfMemory" that runs several tpch queries and injects an
OutOfMemoryException in the ScanBatch. This causes an OUT_OF_MEMORY outcome to be propagated
through the execution stack.
> 
> All unit tests are passing, along with functional and tpch100
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message