drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin" <chriswesti...@gmail.com>
Subject Re: Review Request 34004: DRILL-1942: Improved memory allocator
Date Mon, 22 Jun 2015 19:23:36 GMT


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > common/src/main/java/org/apache/drill/common/config/NestedConfig.java, line 33
> > <https://reviews.apache.org/r/34004/diff/3/?file=983245#file983245line33>
> >
> >     why wrapped?

I'm assuming you mean "Why commented?" Because it's not referenced, and if left uncommented,
it produces a warning.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/codegen/templates/FixedValueVectors.java, line 204
> > <https://reviews.apache.org/r/34004/diff/3/?file=983252#file983252line204>
> >
> >     this isn't necessary given the target.clear() directly above.

That's a remnant from when I was trying to make empty buffers behave properly -- which is
going to have to be a separate major project.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java, line 142
> > <https://reviews.apache.org/r/34004/diff/3/?file=983260#file983260line142>
> >
> >     I generally think this is an excessive optimization throughout the code that
you've added.

Opinion noted; what action do you want me to take? Do you want me to remove them?


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java, line 324
> > <https://reviews.apache.org/r/34004/diff/3/?file=983260#file983260line324>
> >
> >     If we fail to allocate the second vector here, shouldn't we clear the first
(the data vector)

When I rebased, Hanifi had already made that fix.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java, line 74
> > <https://reviews.apache.org/r/34004/diff/3/?file=983261#file983261line74>
> >
> >     With the deprecated items, can you please add javadocs that explain why deprecated
and not removed?

They will be; I kept them for reference. I was just waiting until everything worked.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java, line 78
> > <https://reviews.apache.org/r/34004/diff/3/?file=983265#file983265line78>
> >
> >     Not sure why this would be a system level setting.  It should be a user level
setting.

Neither of the allocation policies we've discussed can be a user-level policy.
(1) Memory is divided equally across all fragments running on a drillbit.
This is the currently implemented policy. This doesn't take users into account. Because of
that, we can't let one user have this setting, and another choose something else (if we had
anything else).
(2) On each drillbit, memory is divided equally across all queries running on that drillbit
(regardless of how many fragments that includes). Note that this also doesn't take users into
account. And it wouldn't work to combine this with (1), unless we also had something higher
level that split things across users in some way. For example, We could say that IF we were
to support splitting memory on a drillbit evenly across all users who have anything running
on that drillbit, then within those arenas, a user could select (1) or (2), but only within
the user's individual arena.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java, line
387
> > <https://reviews.apache.org/r/34004/diff/3/?file=983270#file983270line387>
> >
> >     Why is this necessary?  The debug above would print the exception which would
include the cause.

I don't know. All I know was that I was getting an exception on the unprotected part. Are
you sure that Exception.toString() includes the cause?


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java,
line 59
> > <https://reviews.apache.org/r/34004/diff/3/?file=983272#file983272line59>
> >
> >     Yeah, can we fix this.  Wonder how much memory the tests leak this way?

Submitted DRILL-3335, added that to the TODO.


> On June 18, 2015, 9:40 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java,
line 699
> > <https://reviews.apache.org/r/34004/diff/3/?file=983275#file983275line699>
> >
> >     Can you discuss the change here?  It seems like we should actually just fail
if the length.value is negative.
> >     
> >     Should probably also localize the length.value given multiple references in
perf-sensitive code.

Apparently a negative value was expected some of the time. See the cases down below, which
explicitly test for it. There are similar cases throughout the file. Yeah, it seems wrong
to me, but I don't know the calling code. I don't know why it's like that, but I do know that
the allocator started complaining when I added the checked for negative sizes. (Who knows
what it did before???)

As requested, I localized the length.value in two of the functions that had lots of references.
Side note: you don't think that's excessive optimization, but you do think release(1)/retain(1)
is?


- Chris


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34004/#review88232
-----------------------------------------------------------


On June 16, 2015, 6:07 p.m., Chris Westin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34004/
> -----------------------------------------------------------
> 
> (Updated June 16, 2015, 6:07 p.m.)
> 
> 
> Review request for drill, Jacques Nadeau and Jason Altekruse.
> 
> 
> Bugs: DRILL-1942
>     https://issues.apache.org/jira/browse/DRILL-1942
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Rewritten direct memory allocator. Simplified interface, and use, along with a means
to support additional allocation policies in the future. There are features in the allocator
and in DrillBuf that make finding leaks easier, as well as better enforcement of limits. New
features include transfer of buffers, and better slicing support.
> 
> This is a preliminary patch to get the review started because it touches a lot of files
(readers and record batches were made AutoCloseable in order to cover cleanup). Subsequent
reviews can use the differential view to just see additional changes. The new allocator is
in BaseAllocator.java (along with derived classes RootAllocator and ChildAllocator); DrillBuf
also has significant changes. Most other changes in other files are just to use newer interfaces,
or to change cleanup() to close(), or to close subordinate objects that are newly (Auto)Closeable.
1There are still a couple of things to do:
> * Some TODO(cwestin)s to clean up tracing and debugging code, as well as adding javadoc
> * Using the AllocatorOwner interface to replace the reallocation mechanism for FragmentContext
and OperatorContext so that the allocator doesn't know anything about those objects.
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/AutoCloseablePointer.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillAutoCloseables.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillCloseables.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/HistoricalLog.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/StackTrace.java 54068ec 
>   common/src/main/java/org/apache/drill/common/config/DrillConfig.java 522303f 
>   common/src/main/java/org/apache/drill/common/config/NestedConfig.java 3fd885f 
>   contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java
9458db2 
>   contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java
3c8b9ba 
>   contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
182f5a4 
>   exec/java-exec/src/main/codegen/templates/AbstractFieldWriter.java 1b5dad1 
>   exec/java-exec/src/main/codegen/templates/BaseWriter.java ada410d 
>   exec/java-exec/src/main/codegen/templates/ComplexWriters.java 980f9ac 
>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 0dffa0b 
>   exec/java-exec/src/main/codegen/templates/JsonOutputRecordWriter.java ea643f0 
>   exec/java-exec/src/main/codegen/templates/ListWriters.java ab78603 
>   exec/java-exec/src/main/codegen/templates/MapWriters.java 06a6813 
>   exec/java-exec/src/main/codegen/templates/NullableValueVectors.java ce6a3a7 
>   exec/java-exec/src/main/codegen/templates/ParquetOutputRecordWriter.java 35777b0 
>   exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 37b8fac 
>   exec/java-exec/src/main/codegen/templates/StringOutputRecordWriter.java f704cca 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 529f21b 
>   exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 3ec6b3e 
>   exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java 721aff9 
>   exec/java-exec/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java 2ca79f0 
>   exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java e332b13

>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 8a24e8d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/TestMemoryRetention.java 37e5389

>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/AbstractStreamSerializable.java
ef488d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/LoopedAbstractDrillSerializable.java
d2a7458 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
016cd92 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java c642c4a

>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DumpCat.java 55d9cf3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java
f5a119d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/exception/FragmentSetupException.java
c276846 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/MappifyUtility.java
e27234f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java
49f581f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/Accountor.java ad6a787 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicy.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicyAgent.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationReservation.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorClosedException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorOwner.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorsStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AtomicRemainder.java 057cfa6

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 811cceb

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferLedger.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChainedAllocatorOwner.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChildAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryException.java
063f1c1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryRuntimeException.java
305eabd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocatorStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java e2d5b18

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java c953bb3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 1cbe886

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContext.java 7eb7d8a

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java ce9f351

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 06f8088 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 6176f77

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
76dc91c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
1f6767c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
7e22e65 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
516b028 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
d5d64a7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
e1b5909 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
8af1508 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
b252971 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java
c6a07f8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterRecordBatch.java
5eee9df 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
5fd866f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
ee2ce7f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
2798010 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java
2d37fa5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java
d9330ea 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
baf9bda 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
1286fe1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
31fc160 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
b9a1641 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
dea6ba8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
00f1992 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java
57e7b55 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java
78e83d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
445568b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
684f715 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/validate/IteratorValidatorBatchIterator.java
efd155e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java
428632f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
8871a5f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
9b21ae3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopier.java
161ca6a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
facf192 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
f7cfbf4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
330ec79 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
dd90cab 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java f2f9450

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatch.java 6f10a1c

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java de6f665

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorAccessible.java 9db1681

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 324829a

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector2.java
7a7c012 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector4.java
69bc78f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcDecoder.java 74a4afb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java 80d2d6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryDataBatch.java 914bd00

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
8443948 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserClient.java b39a103

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java d0a998e

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java 0640dbb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractRecordReader.java
6e27628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java 61ccac5

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/avro/AvroRecordReader.java
a52fd22 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
0df6227 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
4d8d4ba 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java
7833631 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java
254e0d8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockRecordReader.java
fd97c48 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockScanBatchCreator.java
74423bf 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java
0cbd480 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java
4e7d628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java
a893da1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java e02b413

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java
c59ade9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
e3a4ba6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java eddefd0

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java
6d356f2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 67c489d

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java f88a7bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/ValueVector.java ab9992e

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractContainerVector.java
d14dca6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
3c01939 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java d0f38c2

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java
b5de8b1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java
a97847b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/DrillBufInputStream.java
1061a5c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
095d8c6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/AbstractBaseWriter.java
ec8c00b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/ComplexWriterImpl.java
a4a35e2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/VectorContainerWriter.java
6b6ab46 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/writer/FieldWriter.java
3faa4f7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java
d52cb5d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
11b6cc8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
cfe5b6b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 5d07b49

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/ForemanException.java
32a99ad 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentManager.java
ad880da 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/NonRootFragmentManager.java
77440c5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
b770a33 
>   exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java a07f621 
>   exec/java-exec/src/test/java/org/apache/drill/PlanTestBase.java f909681 
>   exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java da033f7 
>   exec/java-exec/src/test/java/org/apache/drill/QueryTestUtil.java e218d6c 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java 051ad4e

>   exec/java-exec/src/test/java/org/apache/drill/TestTpchLimit0.java 22471c8 
>   exec/java-exec/src/test/java/org/apache/drill/TestTpchPlanning.java 707ea78 
>   exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java 7c58b19

>   exec/java-exec/src/test/java/org/apache/drill/exec/compile/bytecode/ReplaceMethodInvoke.java
bc2d929 
>   exec/java-exec/src/test/java/org/apache/drill/exec/expr/ExpressionTest.java 239a099

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestByteComparisonFunctions.java
1e9a47c 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java 3ba8743

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestMathFunctions.java 4f06a9d

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNewMathFunctions.java
880184e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java
73c7508 
>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestAllocators.java 74ce225

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestBaseAllocator.java PRE-CREATION

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestEndianess.java 48ddada

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java
42d2193 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestCastFunctions.java
ffa8765 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java
c69c6f5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestImplicitCastFunctions.java
03c6f41 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java
dc37071 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFragmentRun.java
09ba1a5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java
d551319 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestStringFunctions.java
d72c1e1 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestUnionExchange.java
9c24f79 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java d2616a8

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/common/TestHashTable.java
b02249d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java
a069078 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoin.java
6c067fe 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java
18555c7 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java
7cdb41a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/mergereceiver/TestMergingReceiver.java
0122c08 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java
43c430a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java
f37624a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java
b82846e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java
1cb72ff 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java
07de27f 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestWriter.java
f4d505d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestLoad.java f57e765

>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
037c8c6 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/DrillClientFactory.java 4230518

>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
696aed8 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/TestDirectCodecFactory.java
644144e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/ischema/TestInfoSchemaFilterPushDown.java
b6e789b 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/json/TestJsonRecordReader.java
bb1af9e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/FieldInfo.java 34f60ba

>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java
8fdaa72 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
6326478 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestResourceLeak.java d7e317c

>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/TestSplitAndTransfer.java
4b3aa8a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulation.java
06a73e2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulator.java
PRE-CREATION 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/fn/TestJsonReaderWithSparseFiles.java
d674d47 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeReader.java
521a41d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeWriter.java
cb7bef2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestJsonReader.java
912a5f0 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestRepeated.java
6e2a2b5 
>   exec/java-exec/src/test/resources/logback.xml 54ccb42 
>   exec/java-exec/src/test/resources/logback.xml.sav PRE-CREATION 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 7c6ef7e 
> 
> Diff: https://reviews.apache.org/r/34004/diff/
> 
> 
> Testing
> -------
> 
> mvn install passes
> A few tests fail the presubmit suite, but all with the same IndexOutOfBoundsException.
I've just gotten a reproducible case to run standalone on my laptop, so I'm debugging that.
> 
> 
> Thanks,
> 
> Chris Westin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message