drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Westin" <chriswesti...@gmail.com>
Subject Re: Review Request 34004: DRILL-1942: Improved memory allocator
Date Mon, 22 Jun 2015 23:56:08 GMT


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationReservation.java,
line 55
> > <https://reviews.apache.org/r/34004/diff/3/?file=983279#file983279line55>
> >
> >     Preconditions.checkArgument

For this it should be Preconditions.checkState().


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
54
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line54>
> >
> >     should have caps name

I disagree for non-constants.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
304
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line304>
> >
> >     Let's get rid of the int flags.

Disagree. More than one flag can be passed at a time. Calling f(FLAG_X | FLAG_Y) is a lot
more readable than f(true, false, true). (I also think this is better than the Google mandated
solution of creating a separate enum (with XXX_ON and XXX_OFF) for each boolean.)


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
467
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line467>
> >
> >     Why would we grab a global allocator lock if we aren't releasing to the parent?
 This seems rather expensive.  Even if we are releasing to the parent, we should avoid grabbing
the lock for any longer than is necessary.
> >     
> >     In fact, my thought would be that the parent should manage thread safety, not
the child.  Why does the child have to protect the parent.  If this is actually for low variable
use, we should use a local allocator lock rather than a global lock.

I've been through a bunch of variations on this, and the other clever ones all led to problems.
One attempt included per-allocator locks, and that caused a lot of deadlocks as allocations
were shuffled between allocators. In that world, I added code to try to secure all conceivably
required locks for an operation before-hand, in a pre-determined order, and that was a mess;
the possibility of a line of descendant child allocators all having to return things up the
chain to their parent at the time the deepest one released something caused problems. Ownership
transfer and sharing (which tend to work laterally, rather than up and down the allocator
hierarchy) made things worse. This is the simplest scheme I found that works. Given the relative
simplicity of the allocator's work compared to all the stuff around it, contention doesn't
seem to be a problem.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
496
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line496>
> >
> >     Method name is misleading.

Ok, what would you suggest instead?


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
554
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line554>
> >
> >     Same as above, why not?

That will cause getEmpty() to have problems, as an allocator would no longer have it's own
empty buffer; it could end up handing out another allocator's empty buffer, which could lead
to complaints when that allocator is closed, even though the user thought they were using
this allocator. This doesn't appear to cause any problems at this time, so I don't see a reason
to make it work.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
548
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line548>
> >
> >     Why not?

I'm not sure this can't be done, but I don't have a good feeling about it. It doesn't seem
to be required, so I don't see why we should support it at this time. See the comments re
ownership change of the empty buffer.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
575
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line575>
> >
> >     I don't think we need resurrection.  Previously I thought so but upon further
review, it seems overly complicated and unecessary.  Did you find a place where it was required?

It's not just overly complicated and unnecessary, but it's also not really doable. The problem
was described in a comment there -- if a Fragment has been cancelled and cleaned up, and its
allocator is resurrected, there's no handle to that to support closing it again, because its
owning fragment is completely gone.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorsStatsMXBean.java,
line 23
> > <https://reviews.apache.org/r/34004/diff/3/?file=983282#file983282line23>
> >
> >     Let's use the Stats infrastructure rather than introducing an JMX bean

I'll look into this for the next pass.


> On June 21, 2015, 9:06 p.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java, line
119
> > <https://reviews.apache.org/r/34004/diff/3/?file=983284#file983284line119>
> >
> >     Seems like the PerFragmentAllocationPolicy and Agent should be external to the
allocator.

I'm not sure about this -- I had some intuition that some policy implementations might need
to be inner classes. I'll look into it for the next pass.


- Chris


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34004/#review88721
-----------------------------------------------------------


On June 16, 2015, 6:07 p.m., Chris Westin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34004/
> -----------------------------------------------------------
> 
> (Updated June 16, 2015, 6:07 p.m.)
> 
> 
> Review request for drill, Jacques Nadeau and Jason Altekruse.
> 
> 
> Bugs: DRILL-1942
>     https://issues.apache.org/jira/browse/DRILL-1942
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Rewritten direct memory allocator. Simplified interface, and use, along with a means
to support additional allocation policies in the future. There are features in the allocator
and in DrillBuf that make finding leaks easier, as well as better enforcement of limits. New
features include transfer of buffers, and better slicing support.
> 
> This is a preliminary patch to get the review started because it touches a lot of files
(readers and record batches were made AutoCloseable in order to cover cleanup). Subsequent
reviews can use the differential view to just see additional changes. The new allocator is
in BaseAllocator.java (along with derived classes RootAllocator and ChildAllocator); DrillBuf
also has significant changes. Most other changes in other files are just to use newer interfaces,
or to change cleanup() to close(), or to close subordinate objects that are newly (Auto)Closeable.
1There are still a couple of things to do:
> * Some TODO(cwestin)s to clean up tracing and debugging code, as well as adding javadoc
> * Using the AllocatorOwner interface to replace the reallocation mechanism for FragmentContext
and OperatorContext so that the allocator doesn't know anything about those objects.
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/AutoCloseablePointer.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillAutoCloseables.java PRE-CREATION

>   common/src/main/java/org/apache/drill/common/DrillCloseables.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/HistoricalLog.java PRE-CREATION 
>   common/src/main/java/org/apache/drill/common/StackTrace.java 54068ec 
>   common/src/main/java/org/apache/drill/common/config/DrillConfig.java 522303f 
>   common/src/main/java/org/apache/drill/common/config/NestedConfig.java 3fd885f 
>   contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseRecordReader.java
9458db2 
>   contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java
3c8b9ba 
>   contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/MongoRecordReader.java
182f5a4 
>   exec/java-exec/src/main/codegen/templates/AbstractFieldWriter.java 1b5dad1 
>   exec/java-exec/src/main/codegen/templates/BaseWriter.java ada410d 
>   exec/java-exec/src/main/codegen/templates/ComplexWriters.java 980f9ac 
>   exec/java-exec/src/main/codegen/templates/FixedValueVectors.java 0dffa0b 
>   exec/java-exec/src/main/codegen/templates/JsonOutputRecordWriter.java ea643f0 
>   exec/java-exec/src/main/codegen/templates/ListWriters.java ab78603 
>   exec/java-exec/src/main/codegen/templates/MapWriters.java 06a6813 
>   exec/java-exec/src/main/codegen/templates/NullableValueVectors.java ce6a3a7 
>   exec/java-exec/src/main/codegen/templates/ParquetOutputRecordWriter.java 35777b0 
>   exec/java-exec/src/main/codegen/templates/RepeatedValueVectors.java 37b8fac 
>   exec/java-exec/src/main/codegen/templates/StringOutputRecordWriter.java f704cca 
>   exec/java-exec/src/main/codegen/templates/VariableLengthVectors.java 529f21b 
>   exec/java-exec/src/main/java/io/netty/buffer/DrillBuf.java 3ec6b3e 
>   exec/java-exec/src/main/java/io/netty/buffer/FakeAllocator.java 721aff9 
>   exec/java-exec/src/main/java/io/netty/buffer/PooledByteBufAllocatorL.java 2ca79f0 
>   exec/java-exec/src/main/java/io/netty/buffer/UnsafeDirectLittleEndian.java e332b13

>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 8a24e8d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/TestMemoryRetention.java 37e5389

>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/AbstractStreamSerializable.java
ef488d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/LoopedAbstractDrillSerializable.java
d2a7458 
>   exec/java-exec/src/main/java/org/apache/drill/exec/cache/VectorAccessibleSerializable.java
016cd92 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java c642c4a

>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DumpCat.java 55d9cf3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java
f5a119d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/exception/FragmentSetupException.java
c276846 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/MappifyUtility.java
e27234f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/StringFunctions.java
49f581f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/Accountor.java ad6a787 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicy.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationPolicyAgent.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocationReservation.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorClosedException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorOwner.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AllocatorsStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/AtomicRemainder.java 057cfa6

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BaseAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferAllocator.java 811cceb

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/BufferLedger.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChainedAllocatorOwner.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/ChildAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryException.java
063f1c1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/OutOfMemoryRuntimeException.java
305eabd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocator.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/RootAllocatorStatsMXBean.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java e2d5b18

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java c953bb3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 1cbe886

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContext.java 7eb7d8a

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorContextImpl.java ce9f351

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 06f8088 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScanBatch.java 6176f77

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
76dc91c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SingleSenderCreator.java
1f6767c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/PriorityQueueTemplate.java
7e22e65 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/TopN/TopNBatch.java
516b028 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/WriterRecordBatch.java
d5d64a7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
e1b5909 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggTemplate.java
8af1508 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
b252971 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/broadcastsender/BroadcastSenderRootExec.java
c6a07f8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/filter/FilterRecordBatch.java
5eee9df 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/HashJoinBatch.java
5fd866f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatch.java
ee2ce7f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/MergeJoinBatchBuilder.java
2798010 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/join/NestedLoopJoinBatch.java
2d37fa5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/limit/LimitRecordBatch.java
d9330ea 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
baf9bda 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
1286fe1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
31fc160 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
b9a1641 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortBatch.java
dea6ba8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/sort/SortRecordBatchBuilder.java
00f1992 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/svremover/RemovingRecordBatch.java
57e7b55 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/trace/TraceRecordBatch.java
78e83d6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/union/UnionAllRecordBatch.java
445568b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
684f715 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/validate/IteratorValidatorBatchIterator.java
efd155e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/window/WindowFrameRecordBatch.java
428632f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
8871a5f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
9b21ae3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopier.java
161ca6a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/PriorityQueueCopierTemplate.java
facf192 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
f7cfbf4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
330ec79 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractSingleRecordBatch.java
dd90cab 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatch.java f2f9450

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatch.java 6f10a1c

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/RecordBatchLoader.java de6f665

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/VectorAccessible.java 9db1681

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/WritableBatch.java 324829a

>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector2.java
7a7c012 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/selection/SelectionVector4.java
69bc78f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RpcDecoder.java 74a4afb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataServer.java 80d2d6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryDataBatch.java 914bd00

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
8443948 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserClient.java b39a103

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/BootStrapContext.java d0a998e

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java 0640dbb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractRecordReader.java
6e27628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/RecordReader.java 61ccac5

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/avro/AvroRecordReader.java
a52fd22 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JSONRecordReader.java
0df6227 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/JsonProcessor.java
4d8d4ba 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/json/reader/BaseJsonProcessor.java
7833631 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/easy/text/compliant/CompliantTextRecordReader.java
254e0d8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockRecordReader.java
fd97c48 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/mock/MockScanBatchCreator.java
74423bf 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/columnreaders/ParquetRecordReader.java
0cbd480 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet2/DrillParquetReader.java
4e7d628 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/pojo/PojoRecordReader.java
a893da1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java e02b413

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/text/DrillTextRecordReader.java
c59ade9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
e3a4ba6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/AllocationHelper.java eddefd0

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseDataValueVector.java
6d356f2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BaseValueVector.java 67c489d

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/BitVector.java f88a7bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/ValueVector.java ab9992e

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractContainerVector.java
d14dca6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/AbstractMapVector.java
3c01939 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/MapVector.java d0f38c2

>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedListVector.java
b5de8b1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/RepeatedMapVector.java
a97847b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/DrillBufInputStream.java
1061a5c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/fn/JsonReader.java
095d8c6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/AbstractBaseWriter.java
ec8c00b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/ComplexWriterImpl.java
a4a35e2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/impl/VectorContainerWriter.java
6b6ab46 
>   exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/writer/FieldWriter.java
3faa4f7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/AbstractDataCollector.java
d52cb5d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/BaseRawBatchBuffer.java
11b6cc8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
cfe5b6b 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 5d07b49

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/ForemanException.java
32a99ad 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentManager.java
ad880da 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/NonRootFragmentManager.java
77440c5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
b770a33 
>   exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java a07f621 
>   exec/java-exec/src/test/java/org/apache/drill/PlanTestBase.java f909681 
>   exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java da033f7 
>   exec/java-exec/src/test/java/org/apache/drill/QueryTestUtil.java e218d6c 
>   exec/java-exec/src/test/java/org/apache/drill/TestAllocationException.java 051ad4e

>   exec/java-exec/src/test/java/org/apache/drill/TestTpchLimit0.java 22471c8 
>   exec/java-exec/src/test/java/org/apache/drill/TestTpchPlanning.java 707ea78 
>   exec/java-exec/src/test/java/org/apache/drill/exec/client/DumpCatTest.java 7c58b19

>   exec/java-exec/src/test/java/org/apache/drill/exec/compile/bytecode/ReplaceMethodInvoke.java
bc2d929 
>   exec/java-exec/src/test/java/org/apache/drill/exec/expr/ExpressionTest.java 239a099

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestByteComparisonFunctions.java
1e9a47c 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestCastFunctions.java 3ba8743

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestMathFunctions.java 4f06a9d

>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestNewMathFunctions.java
880184e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/impl/TestRepeatedFunction.java
73c7508 
>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestAllocators.java 74ce225

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestBaseAllocator.java PRE-CREATION

>   exec/java-exec/src/test/java/org/apache/drill/exec/memory/TestEndianess.java 48ddada

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java
42d2193 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestCastFunctions.java
ffa8765 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestComparisonFunctions.java
c69c6f5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestImplicitCastFunctions.java
03c6f41 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java
dc37071 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFragmentRun.java
09ba1a5 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestSimpleFunctions.java
d551319 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestStringFunctions.java
d72c1e1 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestUnionExchange.java
9c24f79 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestAgg.java d2616a8

>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/common/TestHashTable.java
b02249d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/filter/TestSimpleFilter.java
a069078 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestHashJoin.java
6c067fe 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/join/TestMergeJoin.java
18555c7 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/limit/TestSimpleLimit.java
7cdb41a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/mergereceiver/TestMergingReceiver.java
0122c08 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/project/TestSimpleProjection.java
43c430a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/sort/TestSimpleSort.java
f37624a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceMultiRecordBatch.java
b82846e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/trace/TestTraceOutputDump.java
1cb72ff 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/union/TestSimpleUnion.java
07de27f 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestWriter.java
f4d505d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestLoad.java f57e765

>   exec/java-exec/src/test/java/org/apache/drill/exec/record/vector/TestValueVector.java
037c8c6 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/DrillClientFactory.java 4230518

>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
696aed8 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/TestDirectCodecFactory.java
644144e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/ischema/TestInfoSchemaFilterPushDown.java
b6e789b 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/json/TestJsonRecordReader.java
bb1af9e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/FieldInfo.java 34f60ba

>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetRecordReaderTest.java
8fdaa72 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
6326478 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestResourceLeak.java d7e317c

>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/TestSplitAndTransfer.java
4b3aa8a 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulation.java
06a73e2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/TestEmptyPopulator.java
PRE-CREATION 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/fn/TestJsonReaderWithSparseFiles.java
d674d47 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeReader.java
521a41d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestComplexTypeWriter.java
cb7bef2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestJsonReader.java
912a5f0 
>   exec/java-exec/src/test/java/org/apache/drill/exec/vector/complex/writer/TestRepeated.java
6e2a2b5 
>   exec/java-exec/src/test/resources/logback.xml 54ccb42 
>   exec/java-exec/src/test/resources/logback.xml.sav PRE-CREATION 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/DrillConnectionImpl.java 7c6ef7e 
> 
> Diff: https://reviews.apache.org/r/34004/diff/
> 
> 
> Testing
> -------
> 
> mvn install passes
> A few tests fail the presubmit suite, but all with the same IndexOutOfBoundsException.
I've just gotten a reproducible case to run standalone on my laptop, so I'm debugging that.
> 
> 
> Thanks,
> 
> Chris Westin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message