drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "abdelhakim deneche" <adene...@gmail.com>
Subject Re: Review Request 33115: DRILL-2762: Update Fragment state reporting and error collection
Date Thu, 16 Apr 2015 22:19:58 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33115/#review80388
-----------------------------------------------------------



common/src/main/java/org/apache/drill/common/DeferredException.java
<https://reviews.apache.org/r/33115/#comment130280>

    Is deferred exception closed at all ? I can't seem to find where this is closed


- abdelhakim deneche


On April 15, 2015, 7 p.m., Jacques Nadeau wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33115/
> -----------------------------------------------------------
> 
> (Updated April 15, 2015, 7 p.m.)
> 
> 
> Review request for drill, abdelhakim deneche, Chris Westin, and Steven Phillips.
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> DRILL-2762: Update Fragment state reporting and error collection
> 
> DeferredException
> - Add new throwAndClear operation on to allow checking for exceptions preClose in FragmentContext
> - Add new getAndClear operation
> 
> BufferManager
> - Ensure close() can be called multiple times by clearing managed buffer list on close().
> 
> FragmentContext/FragmentExecutor
> - Update FragmentContext to have a preClose so that we can check closure state before
doing final close.
> - Update so that there is only a single state maintained between FragmentContext and
FragmentExecutor
> - Clean up FragmentExecutor run() method to better manage error states and have only
single terminal point (avoiding multiple messages to Foreman).
> - Add new CANCELLATION_REQUESTED state for FragmentState.
> - Move all users of isCancelled or isFailed in main code to use shouldContinue()
> - Update receivingFragmentFinished message to not cancel fragment (only inform root operator
of cancellation)
> 
> WorkManager Updates
> - Add new afterExecute command to the WorkManager ExecutorService so that we get log
entries if a thread leaks an exception. (Otherwise logs don't show these exceptions and they
only go to standard out.)
> 
> Foreman/QueryManager
> - Extract listenable interfaces into anonymous inner classes from body of Foreman
> 
> QueryManager
> - Update QueryManager to track completed nodes rather than completed fragments using
NodeTracker
> - Update DrillbitStatusListener to decrement expected completion messages on Nodes that
have died to avoid query hang when a node dies
> 
> FragmentData/MinorFragmentProfile
> - Add ability to track last status update as well as last time fragment made progress
> 
> AbstractRecordBatch
> - Update awareness of current cancellation state to avoid cancellation delays
> 
> Misc. Other changes
> - Move ByteCode optimization code to only record assembly and code as trace messages
> - Update SimpleRootExec to create fake ExecutorState to make existing tests work.
> - Update sort to exit prematurely in the case that the fragment was asked to cancel.
> - Add finals to all edited files.
> - Modify control handler and FragmentManager to directly support receivingFragmentFinished
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/drill/common/DeferredException.java 99f18f1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/compile/AsmUtil.java 81904df 
>   exec/java-exec/src/main/java/org/apache/drill/exec/compile/bytecode/InstructionModifier.java
6c0292e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/compile/bytecode/ScalarReplacementNode.java
6e981bc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/BufferManager.java 2d22d84 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java da2229c

>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/BaseRootExec.java
5b7ca66 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
e230fd2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/producer/ProducerConsumerBatch.java
c50cb8a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/project/ProjectRecordBatch.java
4b317e0 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
389d668 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
a23780e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/MSortTemplate.java
94bc3a3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/proto/helper/QueryIdHelper.java
1aadaa2 
>   exec/java-exec/src/main/java/org/apache/drill/exec/record/AbstractRecordBatch.java
2bb29e5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java c15bb7c 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java e2bcec3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/ControlHandlerImpl.java
3a7123d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java
2430e64 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 23ef0d3

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/FragmentData.java 433ab26

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java 8626d5b

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/AbstractStatusReporter.java
1b0885d 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
a7e6c46 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentManager.java
7a819c4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/NonRootFragmentManager.java
41e87cd 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/RootFragmentManager.java
84071c3 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/StateTransitionException.java
7155d43 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/StatusReporter.java
26b5d68 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/SimpleRootExec.java
933417e 
>   protocol/src/main/java/org/apache/drill/exec/proto/SchemaUserBitShared.java f72d5e1

>   protocol/src/main/java/org/apache/drill/exec/proto/UserBitShared.java 96a921b 
>   protocol/src/main/java/org/apache/drill/exec/proto/beans/FragmentState.java ba536fc

>   protocol/src/main/java/org/apache/drill/exec/proto/beans/MinorFragmentProfile.java
5cd71f9 
>   protocol/src/main/protobuf/UserBitShared.proto 2938114 
> 
> Diff: https://reviews.apache.org/r/33115/diff/
> 
> 
> Testing
> -------
> 
> Regression & Unit, more manual testing planned before final patch.
> 
> 
> Thanks,
> 
> Jacques Nadeau
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message