drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hanifi Gunes" <hanifigu...@gmail.com>
Subject Re: Review Request 33829: DRILL-2757: Verify operators correctly handle low memory conditions and cancellations
Date Wed, 06 May 2015 17:57:21 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33829/#review82691
-----------------------------------------------------------


I reviewed vector changes only. Partial +1 for VV changes.


exec/java-exec/src/main/codegen/templates/FixedValueVectors.java
<https://reviews.apache.org/r/33829/#comment133478>

    Just noticed this thanks to your change. This should fail check style. We disallow single
line if statements with no curly braces. We were able to compile this quite likely because
it is generated code so escapes style check. We should include generated code in style check
as well. In the meantime please add curly braces around return.


- Hanifi Gunes


On May 6, 2015, 4:04 p.m., abdelhakim deneche wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33829/
> -----------------------------------------------------------
> 
> (Updated May 6, 2015, 4:04 p.m.)
> 
> 
> Review request for drill, Hanifi Gunes, Jacques Nadeau, and Steven Phillips.
> 
> 
> Bugs: DRILL-2757
>     https://issues.apache.org/jira/browse/DRILL-2757
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> includes:
> [DRILL-2893](https://issues.apache.org/jira/browse/DRILL-2893): ScanBatch throws a NullPointerException
instead of returning OUT_OF_MEMORY
> [DRILL-2894](https://issues.apache.org/jira/browse/DRILL-2894): FixedValueVectors shouldn't
set it's data buffer to null when it fails to allocate it
> [DRILL-2895](https://issues.apache.org/jira/browse/DRILL-2895): AbstractRecordBatch.buildSchema()
should properly handle OUT_OF_MEMORY outcome
> [DRILL-2905](https://issues.apache.org/jira/browse/DRILL-2905): RootExec implementations
should properly handle IterOutcome.OUT_OF_MEMORY
> [DRILL-2920](https://issues.apache.org/jira/browse/DRILL-2920): properly handle OutOfMemoryException
> [DRILL-2947](https://issues.apache.org/jira/browse/DRILL-2947): AllocationHelper.allocateNew()
doesn't have a consistent behavior when it can't allocate
> 
> also:
> - improved how system errors are displayed
> - added UserException.memoryError() with a pre assigned error message
> - injection site in ScanBatch and unit test that runs various tpch queries and injects
>   an exception in the ScanBatch that will cause an OUT_OF_MEMORY outcome to be sent
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/exceptions/ErrorHelper.java 4da4ee8 
>   common/src/main/java/org/apache/drill/common/exceptions/UserException.java 9283339

>   common/src/test/java/org/apache/drill/common/exceptions/TestUserException.java a145f95

>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 6a924b7 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 8a4b663 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java b108924

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 4700dbd

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
5b4d7bd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
67062f3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
9f6bea9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
15fb7b5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
b753574 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
1b90dd8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
c1c5cb9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggTemplate.java
86f3100 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java
d2282c8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterTemplate2.java
26f2657 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
dd53477 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
6466f70 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java
d20bfa1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java
eff9e61 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
c36b0d3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
cf7ba16 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
35bf3cd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
7b9fffb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
74b7d85 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java
aa9297e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java
af45815 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
d7ea3bb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java
86d11d5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
e88bc67 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
4e348bb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java 7c77ca2

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java 2fc5bf3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java d678cc5

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
ddb828c 
>   exec/java-exec/src/test/java/org/apache/drill/TestOutOfMemoryOutcome.java PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/33829/diff/
> 
> 
> Testing
> -------
> 
> added new test class "TestOutOfMemory" that runs several tpch queries and injects an
OutOfMemoryException in the ScanBatch. This causes an OUT_OF_MEMORY outcome to be propagated
through the execution stack.
> 
> all unit tests are passing along with regression/tpch100
> 
> 
> Thanks,
> 
> abdelhakim deneche
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message