drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam" <skat...@maprtech.com>
Subject Re: Review Request 32795: DRILL-2383: Add exception and pause injections for testing drillbit stability
Date Tue, 21 Apr 2015 00:12:30 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32795/
-----------------------------------------------------------

(Updated April 21, 2015, 12:12 a.m.)


Review request for drill, abdelhakim deneche, Chris Westin, Jacques Nadeau, and Parth Chandra.


Changes
-------

Rebased on current master. 

Changes only in FragmentExecutor
+ Fixed state transition when cancel() before run()


Repository: drill-git


Description
-------

[DRILL-2383](https://issues.apache.org/jira/browse/DRILL-2383): Support to inject exceptions
and pauses in various components of Drill

+ Controls can be introduced in any class that has access to FragmentContext/QueryContext
+ Controls are fired only if assertions are enabled
+ Controls can be fired by altering the DRILLBIT_CONTROL_INJECTIONS session option
+ Renames: SimulatedExceptions => ExecutionControls, ExceptionInjector => ExecutionControlsInjector
+ Added injection sites in Foreman, DrillSqlWorker, FragmentExecutor
+ Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection

Other commits included:

+ [DRILL-2437](https://issues.apache.org/jira/browse/DRILL-2437): Moved ExecutionControls
from DrillbitContext to FragmentContext/QueryContext
+ [DRILL-2382](https://issues.apache.org/jira/browse/DRILL-2382): Added address and port to
Injection to specify drillbit
+ [DRILL-2384](https://issues.apache.org/jira/browse/DRILL-2384): Added QueryState to SingleRowListener
and assert that state is COMPLETED while testing

Other edits:

+ Support for short lived session options in SessionOptionManager (using TTL in OptionValidator)
+ Introduced query count in UserSession
+ Added QueryState to queryCompleted() in UserResultsListener to check if COMPLETED/CANCELED
+ Added JSONStringValidator to TypeValidators
+ Log query id as string in DrillClient, WorkEventBus, QueryResultHandler
+ Use try..catch block only around else clause for OptionList in FragmentContext
+ Fixed drillbitContext spelling error in QueryContext
+ Do not call setLocalOption twice in FallbackOptionManager
+ Show explicitly that submitWork() returns queryId in UserServer
+ Updated protocol/readme.txt to include an alternative way to generate sources


=====
USAGE:

Current checked exception sites: 

+ Foreman: run-try-beginning (ForemanException), run-try-end (ForemanException), send-fragments
(ForemanException)
+ DrillSqlWorker: sql-parsing (ForemanSetupException)
+ FragmentExecutor: fragment-execution (IOException)

Current pause sites:

+ Foreman: pause-run-plan

To set controls:
```
> ALTER SESSION SET `drill.exec.testing.controls`='{
"injections":[
    {
        "type":"exception",
        "siteClass": "org.apache.drill.exec.work.fragment.FragmentExecutor",
        "desc": "fragment-execution",
        "nSkip": 0,
        "nFire": 1,
        "exceptionClass": "java.io.IOException",
        "address": "10.10.10.10",
        "port": 31019
    },
    {
        "type":"pause",
        "siteClass": "org.apache.drill.exec.work.foreman.Foreman",
        "desc": "pause-run-plan",
        "nSkip": 0,
        "nFire": 1,
        "millis": 5000
    }
 ] }';
```
NOTE: 
(1) If controls are specified, they are passed to every fragment as part of PlanFragment.
Then onwards, ExecutionControls live in FragmentContext. And since FragmentContext is created
for every fragment, injections will be fired on ALL fragments.
(2) address and port are optional. If they are set, that injection will be fired ONLY on specified
drillbit. If they are not set, the injection will be fired on EVERY drillbit.


Diffs (updated)
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 7d89ac9 
  exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 336a149 
  exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java 2bf35b1

  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java c46613d 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 2fa0b18 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java 6b3caf4

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java 9ca64d8

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java a5a5441

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java 3c807d5

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java f928476

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java 877bc08 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java 19d77b0 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java dbf3c74 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java
4e90616 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java 0b8811a

  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionValidator.java 43071e7

  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java
c3de190 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java
4471d4f 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java b9721cc

  exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java 62f5bdb

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjection.java 68cbf08

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjector.java 54bc351

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControls.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/Injection.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionConfigurationException.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionSite.java 9e19fdd 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/PauseInjection.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/SimulatedExceptions.java 0292c08

  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java d94b9f0 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java 3570ba5

  exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java 854f474 
  exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java d8ed0b3 
  exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java 264123f 
  exec/java-exec/src/test/java/org/apache/drill/SingleRowListener.java 99aa9fc 
  exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestClassTransformation.java
f5f5b8d 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java 7aee6d3

  exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
3a794a9 
  exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetPhysicalPlan.java
cfe52c2 
  exec/java-exec/src/test/java/org/apache/drill/exec/testing/ControlsInjectionUtil.java PRE-CREATION

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/ExceptionInjectionUtil.java bf93dee

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestExceptionInjection.java d0c0279

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestPauseInjection.java PRE-CREATION

  exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillResultSetImpl.java d2302fb 
  protocol/readme.txt bd516d3 

Diff: https://reviews.apache.org/r/32795/diff/


Testing
-------

Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection.
Successful Jenkins builds.


Thanks,

Sudheesh Katkam


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message