drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau" <jacques.dr...@gmail.com>
Subject Re: Review Request 34004: DRILL-1942: Improved memory allocator
Date Thu, 18 Jun 2015 16:40:50 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34004/#review88232
-----------------------------------------------------------


Feedback on the first two pages of changes.


common/src/main/java/org/apache/drill/common/config/DrillConfig.java (line 59)
<https://reviews.apache.org/r/34004/#comment140645>

    I think this is old and can just be entirely deleted.



common/src/main/java/org/apache/drill/common/config/NestedConfig.java (line 33)
<https://reviews.apache.org/r/34004/#comment140655>

    why wrapped?



exec/java-exec/src/main/codegen/templates/FixedValueVectors.java (line 203)
<https://reviews.apache.org/r/34004/#comment140661>

    this isn't necessary given the target.clear() directly above.



exec/java-exec/src/main/codegen/templates/FixedValueVectors.java (line 215)
<https://reviews.apache.org/r/34004/#comment140660>

    why not use clear here?  it seems nicely encapsulated.



exec/java-exec/src/main/codegen/templates/FixedValueVectors.java (line 224)
<https://reviews.apache.org/r/34004/#comment140662>

    private?



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 137)
<https://reviews.apache.org/r/34004/#comment140865>

    I generally think this is an excessive optimization throughout the code that you've added.



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 205)
<https://reviews.apache.org/r/34004/#comment140866>

    Shouldn't we switch this to target.clear() rather than reaching inside?



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 308)
<https://reviews.apache.org/r/34004/#comment140867>

    Why is this necessary?  We already called clear at the beginning of this method.



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 314)
<https://reviews.apache.org/r/34004/#comment140868>

    If we fail to allocate the second vector here, shouldn't we clear the first (the data
vector)



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 330)
<https://reviews.apache.org/r/34004/#comment140869>

    same as above.  Why isn't clear enough?



exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java (line 492)
<https://reviews.apache.org/r/34004/#comment140870>

    Can you please update this.  We shouldn't be retrieving the accessor over and over in
every method.  We should be grabbing it at class initialization and then referencing it directly.



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 61)
<https://reviews.apache.org/r/34004/#comment140871>

    the comment seems to be out of sync with the code/location of fields.



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 70)
<https://reviews.apache.org/r/34004/#comment140872>

    With the deprecated items, can you please add javadocs that explain why deprecated and
not removed?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 74)
<https://reviews.apache.org/r/34004/#comment140873>

    same deprecated, please add javadoc



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 75)
<https://reviews.apache.org/r/34004/#comment140874>

    same deprecated, please add javadoc



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 120)
<https://reviews.apache.org/r/34004/#comment140875>

    please add javadoc



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 173)
<https://reviews.apache.org/r/34004/#comment140876>

    Shouldn't this be entirely removed by your code changes?  We should be able to change
that code to use shareOwnership(), right?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 187)
<https://reviews.apache.org/r/34004/#comment140877>

    javadoc, why not use java enum?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 218)
<https://reviews.apache.org/r/34004/#comment140878>

    Won't this go in an infinite loop if we use the EmptyByteBuffer path?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 244)
<https://reviews.apache.org/r/34004/#comment140879>

    not sure what "buffer argument may go away" means.



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 302)
<https://reviews.apache.org/r/34004/#comment140880>

    why not make it part of the constructor?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 312)
<https://reviews.apache.org/r/34004/#comment140881>

    Since we have a clean approach to the underlying buffers now, can we modify by this so
the return is always (return == this) is always true?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 322)
<https://reviews.apache.org/r/34004/#comment140882>

    For simplicity, let's just have fragmentContext and and operatorContext hold a private
BufferManager implementation instance that they can use.



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 399)
<https://reviews.apache.org/r/34004/#comment140883>

    Why do need the allocator and the ledger?  Does one automatically imply the other?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 418)
<https://reviews.apache.org/r/34004/#comment140884>

    If this is used by the BufferAllocator, it seems like it shouldn't be public to the world.



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 455)
<https://reviews.apache.org/r/34004/#comment140885>

    Preconditions.checkArgument()



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 568)
<https://reviews.apache.org/r/34004/#comment140886>

    Isn't that now?



exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java (line 1022)
<https://reviews.apache.org/r/34004/#comment140887>

    Does this need to be public?



exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java (line 28)
<https://reviews.apache.org/r/34004/#comment140888>

    Do we still need it?



exec/java-exec/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java (line 178)
<https://reviews.apache.org/r/34004/#comment140889>

    Preconditions.checkArgument



exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java (line 37)
<https://reviews.apache.org/r/34004/#comment140890>

    Why not just have another static flag in this class that enables/disables this functionality.
 As it is (commented out), I'm sure it will get out of sync very quickly.



exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java (line 138)
<https://reviews.apache.org/r/34004/#comment140891>

    delete



exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java (line 139)
<https://reviews.apache.org/r/34004/#comment140892>

    we can just return here, no need to store.



exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java (line 186)
<https://reviews.apache.org/r/34004/#comment140893>

    same, let's just return



exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java (line 198)
<https://reviews.apache.org/r/34004/#comment140894>

    same, let's just return



exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java (line 77)
<https://reviews.apache.org/r/34004/#comment140895>

    this doesn't match the style of other options.  Maybe enable-debugging or debug.enabled?



exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java (line 78)
<https://reviews.apache.org/r/34004/#comment140896>

    Not sure why this would be a system level setting.  It should be a user level setting.



exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java (line 385)
<https://reviews.apache.org/r/34004/#comment140898>

    Why is this necessary?  The debug above would print the exception which would include
the cause.



exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java (line
59)
<https://reviews.apache.org/r/34004/#comment140899>

    Yeah, can we fix this.  Wonder how much memory the tests leak this way?



exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java (line
699)
<https://reviews.apache.org/r/34004/#comment140901>

    Can you discuss the change here?  It seems like we should actually just fail if the length.value
is negative.
    
    Should probably also localize the length.value given multiple references in perf-sensitive
code.


- Jacques Nadeau


On June 17, 2015, 1:07 a.m., Chris Westin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34004/
> -----------------------------------------------------------
> 
> (Updated June 17, 2015, 1:07 a.m.)
> 
> 
> Review request for drill, Jacques Nadeau and Jason Altekruse.
> 
> 
> Bugs: DRILL-1942
>     https://issues.apache.org/jira/browse/DRILL-1942
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Rewritten direct memory allocator. Simplified interface, and use, along with a means
to support additional allocation policies in the future. There are features in the allocator
and in DrillBuf that make finding leaks easier, as well as better enforcement of limits. New
features include transfer of buffers, and better slicing support.
> 
> This is a preliminary patch to get the review started because it touches a lot of files
(readers and record batches were made AutoCloseable in order to cover cleanup). Subsequent
reviews can use the differential view to just see additional changes. The new allocator is
in BaseAllocator.java (along with derived classes RootAllocator and ChildAllocator); DrillBuf
also has significant changes. Most other changes in other files are just to use newer interfaces,
or to change cleanup() to close(), or to close subordinate objects that are newly (Auto)Closeable.
1There are still a couple of things to do:
> * Some TODO(cwestin)s to clean up tracing and debugging code, as well as adding javadoc
> * Using the AllocatorOwner interface to replace the reallocation mechanism for FragmentContext
and OperatorContext so that the allocator doesn't know anything about those objects.
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/AutoCloseablePointer.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillAutoCloseables.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillCloseables.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/HistoricalLog.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/StackTrace.java 54068ec 
>   common/src/main/java/org/apache/drill/common/config/DrillConfig.java 522303f 
>   common/src/main/java/org/apache/drill/common/config/NestedConfig.java 3fd885f 
>   contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java
9458db2 
>   contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java
3c8b9ba 
>   contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
182f5a4 
>   exec/java-exec/src/main/codegen/templates/AbstractFieldWriter.java 1b5dad1 
>   exec/java-exec/src/main/codegen/templates/BaseWriter.java ada410d 
>   exec/java-exec/src/main/codegen/templates/ComplexWriters.java 980f9ac 
>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 0dffa0b 
>   exec/java-exec/src/main/codegen/templates/JsonOutputRecordWriter.java ea643f0 
>   exec/java-exec/src/main/codegen/templates/ListWriters.java ab78603 
>   exec/java-exec/src/main/codegen/templates/MapWriters.java 06a6813 
>   exec/java-exec/src/main/codegen/templates/NullableValueVectors.java ce6a3a7 
>   exec/java-exec/src/main/codegen/templates/ParquetOutputRecordWriter.java 35777b0 
>   exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 37b8fac 
>   exec/java-exec/src/main/codegen/templates/StringOutputRecordWriter.java f704cca 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 529f21b 
>   exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 3ec6b3e 
>   exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java 721aff9 
>   exec/java-exec/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java 2ca79f0 
>   exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java e332b13

>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 8a24e8d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/TestMemoryRetention.java 37e5389

>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/AbstractStreamSerializable.java
ef488d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/LoopedAbstractDrillSerializable.java
d2a7458 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
016cd92 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java c642c4a

>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DumpCat.java 55d9cf3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java
f5a119d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/exception/FragmentSetupException.java
c276846 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/MappifyUtility.java
e27234f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java
49f581f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/Accountor.java ad6a787 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicy.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicyAgent.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationReservation.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorClosedException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorOwner.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorsStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AtomicRemainder.java 057cfa6

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 811cceb

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferLedger.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChainedAllocatorOwner.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChildAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryException.java
063f1c1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryRuntimeException.java
305eabd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocatorStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java e2d5b18

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java c953bb3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 1cbe886

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContext.java 7eb7d8a

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java ce9f351

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 06f8088 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 6176f77

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
76dc91c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
1f6767c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
7e22e65 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
516b028 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
d5d64a7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
e1b5909 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
8af1508 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
b252971 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java
c6a07f8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterRecordBatch.java
5eee9df 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
5fd866f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
ee2ce7f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
2798010 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java
2d37fa5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java
d9330ea 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
baf9bda 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
1286fe1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
31fc160 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
b9a1641 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
dea6ba8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
00f1992 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java
57e7b55 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java
78e83d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
445568b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
684f715 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/validate/IteratorValidatorBatchIterator.java
efd155e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java
428632f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
8871a5f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
9b21ae3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopier.java
161ca6a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
facf192 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
f7cfbf4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
330ec79 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
dd90cab 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java f2f9450

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatch.java 6f10a1c

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java de6f665

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorAccessible.java 9db1681

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 324829a

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector2.java
7a7c012 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector4.java
69bc78f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcDecoder.java 74a4afb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java 80d2d6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryDataBatch.java 914bd00

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
8443948 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserClient.java b39a103

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java d0a998e

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java 0640dbb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractRecordReader.java
6e27628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java 61ccac5

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/avro/AvroRecordReader.java
a52fd22 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
0df6227 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
4d8d4ba 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java
7833631 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java
254e0d8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockRecordReader.java
fd97c48 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockScanBatchCreator.java
74423bf 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java
0cbd480 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java
4e7d628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java
a893da1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java e02b413

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java
c59ade9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
e3a4ba6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java eddefd0

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java
6d356f2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 67c489d

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java f88a7bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/ValueVector.java ab9992e

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractContainerVector.java
d14dca6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
3c01939 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java d0f38c2

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java
b5de8b1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java
a97847b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/DrillBufInputStream.java
1061a5c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
095d8c6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/AbstractBaseWriter.java
ec8c00b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/ComplexWriterImpl.java
a4a35e2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/VectorContainerWriter.java
6b6ab46 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/writer/FieldWriter.java
3faa4f7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java
d52cb5d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
11b6cc8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
cfe5b6b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 5d07b49

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/ForemanException.java
32a99ad 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentManager.java
ad880da 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/NonRootFragmentManager.java
77440c5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
b770a33 
>   exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java a07f621 
>   exec/java-exec/src/test/java/org/apache/drill/PlanTestBase.java f909681 
>   exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java da033f7 
>   exec/java-exec/src/test/java/org/apache/drill/QueryTestUtil.java e218d6c 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java 051ad4e

>   exec/java-exec/src/test/java/org/apache/drill/TestTpchLimit0.java 22471c8 
>   exec/java-exec/src/test/java/org/apache/drill/TestTpchPlanning.java 707ea78 
>   exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java 7c58b19

>   exec/java-exec/src/test/java/org/apache/drill/exec/compile/bytecode/ReplaceMethodInvoke.java
bc2d929 
>   exec/java-exec/src/test/java/org/apache/drill/exec/expr/ExpressionTest.java 239a099

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestByteComparisonFunctions.java
1e9a47c 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java 3ba8743

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestMathFunctions.java 4f06a9d

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNewMathFunctions.java
880184e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java
73c7508 
>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestAllocators.java 74ce225

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestBaseAllocator.java PRE-CREATION

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestEndianess.java 48ddada

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java
42d2193 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestCastFunctions.java
ffa8765 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java
c69c6f5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestImplicitCastFunctions.java
03c6f41 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java
dc37071 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFragmentRun.java
09ba1a5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java
d551319 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestStringFunctions.java
d72c1e1 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestUnionExchange.java
9c24f79 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java d2616a8

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/common/TestHashTable.java
b02249d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java
a069078 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoin.java
6c067fe 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java
18555c7 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java
7cdb41a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/mergereceiver/TestMergingReceiver.java
0122c08 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java
43c430a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java
f37624a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java
b82846e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java
1cb72ff 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java
07de27f 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestWriter.java
f4d505d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestLoad.java f57e765

>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
037c8c6 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/DrillClientFactory.java 4230518

>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
696aed8 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/TestDirectCodecFactory.java
644144e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/ischema/TestInfoSchemaFilterPushDown.java
b6e789b 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/json/TestJsonRecordReader.java
bb1af9e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/FieldInfo.java 34f60ba

>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java
8fdaa72 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
6326478 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestResourceLeak.java d7e317c

>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/TestSplitAndTransfer.java
4b3aa8a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulation.java
06a73e2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulator.java
PRE-CREATION 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/fn/TestJsonReaderWithSparseFiles.java
d674d47 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeReader.java
521a41d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeWriter.java
cb7bef2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestJsonReader.java
912a5f0 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestRepeated.java
6e2a2b5 
>   exec/java-exec/src/test/resources/logback.xml 54ccb42 
>   exec/java-exec/src/test/resources/logback.xml.sav PRE-CREATION 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 7c6ef7e 
> 
> Diff: https://reviews.apache.org/r/34004/diff/
> 
> 
> Testing
> -------
> 
> mvn install passes
> A few tests fail the presubmit suite, but all with the same IndexOutOfBoundsException.
I've just gotten a reproducible case to run standalone on my laptop, so I'm debugging that.
> 
> 
> Thanks,
> 
> Chris Westin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message