drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sudheesh Katkam" <skat...@maprtech.com>
Subject Re: Review Request 32795: DRILL-2383: Add exception and pause injections for testing drillbit stability
Date Wed, 15 Apr 2015 16:45:13 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32795/
-----------------------------------------------------------

(Updated April 15, 2015, 4:45 p.m.)


Review request for drill, abdelhakim deneche, Chris Westin, and Jacques Nadeau.


Changes
-------

Addressing Chris' comments and rebased on current master.

+ Changed OptionValidator interface so that validators for short-lived options need to override
two methods.


Repository: drill-git


Description
-------

[DRILL-2383](https://issues.apache.org/jira/browse/DRILL-2383): Support to inject exceptions
and pauses in various components of Drill

+ Controls can be introduced in any class that has access to FragmentContext/QueryContext
+ Controls are fired only if assertions are enabled
+ Controls can be fired by altering the DRILLBIT_CONTROL_INJECTIONS session option
+ Renames: SimulatedExceptions => ExecutionControls, ExceptionInjector => ExecutionControlsInjector
+ Added injection sites in Foreman, DrillSqlWorker, FragmentExecutor
+ Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection

Other commits included:

+ [DRILL-2437](https://issues.apache.org/jira/browse/DRILL-2437): Moved ExecutionControls
from DrillbitContext to FragmentContext/QueryContext
+ [DRILL-2382](https://issues.apache.org/jira/browse/DRILL-2382): Added address and port to
Injection to specify drillbit
+ [DRILL-2384](https://issues.apache.org/jira/browse/DRILL-2384): Added QueryState to SingleRowListener
and assert that state is COMPLETED while testing

Other edits:

+ Support for short lived session options in SessionOptionManager (using TTL in OptionValidator)
+ Introduced query count in UserSession
+ Added QueryState to queryCompleted() in UserResultsListener to check if COMPLETED/CANCELED
+ Added JSONStringValidator to TypeValidators
+ Log query id as string in DrillClient, WorkEventBus, QueryResultHandler
+ Use try..catch block only around else clause for OptionList in FragmentContext
+ Fixed drillbitContext spelling error in QueryContext
+ Do not call setLocalOption twice in FallbackOptionManager
+ Show explicitly that submitWork() returns queryId in UserServer
+ Updated protocol/readme.txt to include an alternative way to generate sources


=====
USAGE:

Current checked exception sites: 

+ Foreman: run-try-beginning (ForemanException), run-try-end (ForemanException), send-fragments
(ForemanException)
+ DrillSqlWorker: sql-parsing (ForemanSetupException)
+ FragmentExecutor: fragment-execution (IOException)

Current pause sites:

+ Foreman: pause-run-plan

To set controls:
```
> ALTER SESSION SET `drill.exec.testing.controls`='{
"injections":[
    {
        "type":"exception",
        "siteClass": "org.apache.drill.exec.work.fragment.FragmentExecutor",
        "desc": "fragment-execution",
        "nSkip": 0,
        "nFire": 1,
        "exceptionClass": "java.io.IOException",
        "address": "10.10.10.10",
        "port": 31019
    },
    {
        "type":"pause",
        "siteClass": "org.apache.drill.exec.work.foreman.Foreman",
        "desc": "pause-run-plan",
        "nSkip": 0,
        "nFire": 1,
        "millis": 5000
    }
 ] }';
```
NOTE: 
(1) If controls are specified, they are passed to every fragment as part of PlanFragment.
Then onwards, ExecutionControls live in FragmentContext. And since FragmentContext is created
for every fragment, injections will be fired on ALL fragments.
(2) address and port are optional. If they are set, that injection will be fired ONLY on specified
drillbit. If they are not set, the injection will be fired on EVERY drillbit.


Diffs (updated)
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 7d89ac9 
  exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 6555cad 
  exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java a11cec0

  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 8c402e8 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 2fa0b18 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java 7892999

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java a5a5441

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java a03e2c0

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java b9f742c

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java 877bc08 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java 19d77b0 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java dbf3c74 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java
4e90616 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java 0b8811a

  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionValidator.java 43071e7

  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java
c3de190 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java
4471d4f 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java b9721cc

  exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java 8415440

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjection.java 68cbf08

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjector.java 54bc351

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControls.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/Injection.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionConfigurationException.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionSite.java 9e19fdd 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/PauseInjection.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/SimulatedExceptions.java 0292c08

  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 0df5145 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java 58f4256

  exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java 854f474 
  exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java 6df5801 
  exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java 264123f 
  exec/java-exec/src/test/java/org/apache/drill/SingleRowListener.java 715904d 
  exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestClassTransformation.java
f5f5b8d 
  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java 69c274c

  exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java
3bcfbdf 
  exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetPhysicalPlan.java
cde2e95 
  exec/java-exec/src/test/java/org/apache/drill/exec/testing/ControlsInjectionUtil.java PRE-CREATION

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/ExceptionInjectionUtil.java bf93dee

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestExceptionInjection.java d0c0279

  exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestPauseInjection.java PRE-CREATION

  exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillResultSetImpl.java a9753bc 
  protocol/readme.txt bd516d3 

Diff: https://reviews.apache.org/r/32795/diff/


Testing
-------

Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection.
Successful Jenkins builds.


Thanks,

Sudheesh Katkam


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message