drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam" <skat...@maprtech.com>
Subject Re: Review Request 32795: DRILL-2383: Add exception and pause injections for testing drillbit stability
Date Fri, 17 Apr 2015 18:24:31 GMT


> On April 17, 2015, 5:50 p.m., Parth Chandra wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java,
line 44
> > <https://reviews.apache.org/r/32795/diff/6/?file=933949#file933949line44>
> >
> >     Are you using the queryState parameter anywhere? I didn't see this param being
used in any of the implementations.

The QueryState is used in testing (see TestDrillbitResilience and SingleRowListener) since
queryCompleted() could mean QueryState is COMPLETED or CANCELED.


- Sudheesh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32795/#review80488
-----------------------------------------------------------


On April 17, 2015, 5:28 p.m., Sudheesh Katkam wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32795/
> -----------------------------------------------------------
> 
> (Updated April 17, 2015, 5:28 p.m.)
> 
> 
> Review request for drill, abdelhakim deneche, Chris Westin, and Jacques Nadeau.
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> [DRILL-2383](https://issues.apache.org/jira/browse/DRILL-2383): Support to inject exceptions
and pauses in various components of Drill
> 
> + Controls can be introduced in any class that has access to FragmentContext/QueryContext
> + Controls are fired only if assertions are enabled
> + Controls can be fired by altering the DRILLBIT_CONTROL_INJECTIONS session option
> + Renames: SimulatedExceptions => ExecutionControls, ExceptionInjector => ExecutionControlsInjector
> + Added injection sites in Foreman, DrillSqlWorker, FragmentExecutor
> + Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection
> 
> Other commits included:
> 
> + [DRILL-2437](https://issues.apache.org/jira/browse/DRILL-2437): Moved ExecutionControls
from DrillbitContext to FragmentContext/QueryContext
> + [DRILL-2382](https://issues.apache.org/jira/browse/DRILL-2382): Added address and port
to Injection to specify drillbit
> + [DRILL-2384](https://issues.apache.org/jira/browse/DRILL-2384): Added QueryState to
SingleRowListener and assert that state is COMPLETED while testing
> 
> Other edits:
> 
> + Support for short lived session options in SessionOptionManager (using TTL in OptionValidator)
> + Introduced query count in UserSession
> + Added QueryState to queryCompleted() in UserResultsListener to check if COMPLETED/CANCELED
> + Added JSONStringValidator to TypeValidators
> + Log query id as string in DrillClient, WorkEventBus, QueryResultHandler
> + Use try..catch block only around else clause for OptionList in FragmentContext
> + Fixed drillbitContext spelling error in QueryContext
> + Do not call setLocalOption twice in FallbackOptionManager
> + Show explicitly that submitWork() returns queryId in UserServer
> + Updated protocol/readme.txt to include an alternative way to generate sources
> 
> 
> =====
> USAGE:
> 
> Current checked exception sites: 
> 
> + Foreman: run-try-beginning (ForemanException), run-try-end (ForemanException), send-fragments
(ForemanException)
> + DrillSqlWorker: sql-parsing (ForemanSetupException)
> + FragmentExecutor: fragment-execution (IOException)
> 
> Current pause sites:
> 
> + Foreman: pause-run-plan
> 
> To set controls:
> ```
> > ALTER SESSION SET `drill.exec.testing.controls`='{
> "injections":[
>     {
>         "type":"exception",
>         "siteClass": "org.apache.drill.exec.work.fragment.FragmentExecutor",
>         "desc": "fragment-execution",
>         "nSkip": 0,
>         "nFire": 1,
>         "exceptionClass": "java.io.IOException",
>         "address": "10.10.10.10",
>         "port": 31019
>     },
>     {
>         "type":"pause",
>         "siteClass": "org.apache.drill.exec.work.foreman.Foreman",
>         "desc": "pause-run-plan",
>         "nSkip": 0,
>         "nFire": 1,
>         "millis": 5000
>     }
>  ] }';
> ```
> NOTE: 
> (1) If controls are specified, they are passed to every fragment as part of PlanFragment.
Then onwards, ExecutionControls live in FragmentContext. And since FragmentContext is created
for every fragment, injections will be fired on ALL fragments.
> (2) address and port are optional. If they are set, that injection will be fired ONLY
on specified drillbit. If they are not set, the injection will be fired on EVERY drillbit.
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 7d89ac9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 336a149

>   exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java
2bf35b1 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 44ca78a

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 2fa0b18 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java
6b3caf4 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java
9ca64d8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java a5a5441

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java
3c807d5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java
f928476 
>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java 877bc08

>   exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java 19d77b0

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java dbf3c74

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java
4e90616 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java
0b8811a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionValidator.java
43071e7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java
c3de190 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java
4471d4f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java
b9721cc 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java 62f5bdb

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjection.java
68cbf08 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjector.java 54bc351

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControls.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/Injection.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionConfigurationException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionSite.java 9e19fdd

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/PauseInjection.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/testing/SimulatedExceptions.java
0292c08 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java f824b53

>   exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java
a4a97c9 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java 854f474

>   exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java 3931039 
>   exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java 264123f 
>   exec/java-exec/src/test/java/org/apache/drill/SingleRowListener.java 99aa9fc 
>   exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestClassTransformation.java
f5f5b8d 
>   exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java
7aee6d3 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
3a794a9 
>   exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetPhysicalPlan.java
cfe52c2 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/ControlsInjectionUtil.java
PRE-CREATION 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/ExceptionInjectionUtil.java
bf93dee 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestExceptionInjection.java
d0c0279 
>   exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestPauseInjection.java
PRE-CREATION 
>   exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillResultSetImpl.java d2302fb

>   protocol/readme.txt bd516d3 
> 
> Diff: https://reviews.apache.org/r/32795/diff/
> 
> 
> Testing
> -------
> 
> Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection.
> Successful Jenkins builds.
> 
> 
> Thanks,
> 
> Sudheesh Katkam
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message