Return-Path: X-Original-To: apmail-drill-dev-archive@www.apache.org Delivered-To: apmail-drill-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0D45017795 for ; Fri, 17 Apr 2015 18:24:30 +0000 (UTC) Received: (qmail 52329 invoked by uid 500); 17 Apr 2015 18:24:29 -0000 Delivered-To: apmail-drill-dev-archive@drill.apache.org Received: (qmail 52279 invoked by uid 500); 17 Apr 2015 18:24:29 -0000 Mailing-List: contact dev-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list dev@drill.apache.org Received: (qmail 52267 invoked by uid 500); 17 Apr 2015 18:24:29 -0000 Delivered-To: apmail-incubator-drill-dev@incubator.apache.org Received: (qmail 52259 invoked by uid 99); 17 Apr 2015 18:24:29 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 17 Apr 2015 18:24:29 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 4C7851DB2D5; Fri, 17 Apr 2015 18:24:31 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============3387930206556218284==" MIME-Version: 1.0 Subject: Re: Review Request 32795: DRILL-2383: Add exception and pause injections for testing drillbit stability From: "Sudheesh Katkam" To: "Jacques Nadeau" , "Chris Westin" , "abdelhakim deneche" Cc: "drill" , "Sudheesh Katkam" , "Parth Chandra" Date: Fri, 17 Apr 2015 18:24:31 -0000 Message-ID: <20150417182431.1422.40825@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: "Sudheesh Katkam" X-ReviewGroup: drill-git X-ReviewRequest-URL: https://reviews.apache.org/r/32795/ X-Sender: "Sudheesh Katkam" References: <20150417175018.1421.97154@reviews.apache.org> In-Reply-To: <20150417175018.1421.97154@reviews.apache.org> Reply-To: "Sudheesh Katkam" X-ReviewRequest-Repository: drill-git --===============3387930206556218284== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit > On April 17, 2015, 5:50 p.m., Parth Chandra wrote: > > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java, line 44 > > > > > > Are you using the queryState parameter anywhere? I didn't see this param being used in any of the implementations. The QueryState is used in testing (see TestDrillbitResilience and SingleRowListener) since queryCompleted() could mean QueryState is COMPLETED or CANCELED. - Sudheesh ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/32795/#review80488 ----------------------------------------------------------- On April 17, 2015, 5:28 p.m., Sudheesh Katkam wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/32795/ > ----------------------------------------------------------- > > (Updated April 17, 2015, 5:28 p.m.) > > > Review request for drill, abdelhakim deneche, Chris Westin, and Jacques Nadeau. > > > Repository: drill-git > > > Description > ------- > > [DRILL-2383](https://issues.apache.org/jira/browse/DRILL-2383): Support to inject exceptions and pauses in various components of Drill > > + Controls can be introduced in any class that has access to FragmentContext/QueryContext > + Controls are fired only if assertions are enabled > + Controls can be fired by altering the DRILLBIT_CONTROL_INJECTIONS session option > + Renames: SimulatedExceptions => ExecutionControls, ExceptionInjector => ExecutionControlsInjector > + Added injection sites in Foreman, DrillSqlWorker, FragmentExecutor > + Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection > > Other commits included: > > + [DRILL-2437](https://issues.apache.org/jira/browse/DRILL-2437): Moved ExecutionControls from DrillbitContext to FragmentContext/QueryContext > + [DRILL-2382](https://issues.apache.org/jira/browse/DRILL-2382): Added address and port to Injection to specify drillbit > + [DRILL-2384](https://issues.apache.org/jira/browse/DRILL-2384): Added QueryState to SingleRowListener and assert that state is COMPLETED while testing > > Other edits: > > + Support for short lived session options in SessionOptionManager (using TTL in OptionValidator) > + Introduced query count in UserSession > + Added QueryState to queryCompleted() in UserResultsListener to check if COMPLETED/CANCELED > + Added JSONStringValidator to TypeValidators > + Log query id as string in DrillClient, WorkEventBus, QueryResultHandler > + Use try..catch block only around else clause for OptionList in FragmentContext > + Fixed drillbitContext spelling error in QueryContext > + Do not call setLocalOption twice in FallbackOptionManager > + Show explicitly that submitWork() returns queryId in UserServer > + Updated protocol/readme.txt to include an alternative way to generate sources > > > ===== > USAGE: > > Current checked exception sites: > > + Foreman: run-try-beginning (ForemanException), run-try-end (ForemanException), send-fragments (ForemanException) > + DrillSqlWorker: sql-parsing (ForemanSetupException) > + FragmentExecutor: fragment-execution (IOException) > > Current pause sites: > > + Foreman: pause-run-plan > > To set controls: > ``` > > ALTER SESSION SET `drill.exec.testing.controls`='{ > "injections":[ > { > "type":"exception", > "siteClass": "org.apache.drill.exec.work.fragment.FragmentExecutor", > "desc": "fragment-execution", > "nSkip": 0, > "nFire": 1, > "exceptionClass": "java.io.IOException", > "address": "10.10.10.10", > "port": 31019 > }, > { > "type":"pause", > "siteClass": "org.apache.drill.exec.work.foreman.Foreman", > "desc": "pause-run-plan", > "nSkip": 0, > "nFire": 1, > "millis": 5000 > } > ] }'; > ``` > NOTE: > (1) If controls are specified, they are passed to every fragment as part of PlanFragment. Then onwards, ExecutionControls live in FragmentContext. And since FragmentContext is created for every fragment, injections will be fired on ALL fragments. > (2) address and port are optional. If they are set, that injection will be fired ONLY on specified drillbit. If they are not set, the injection will be fired on EVERY drillbit. > > > Diffs > ----- > > exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java 7d89ac9 > exec/java-exec/src/main/java/org/apache/drill/exec/client/DrillClient.java 336a149 > exec/java-exec/src/main/java/org/apache/drill/exec/client/PrintingResultsListener.java 2bf35b1 > exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 44ca78a > exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 2fa0b18 > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/ScreenCreator.java 6b3caf4 > exec/java-exec/src/main/java/org/apache/drill/exec/planner/sql/DrillSqlWorker.java 9ca64d8 > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/control/WorkEventBus.java a5a5441 > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/QueryResultHandler.java 3c807d5 > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserResultsListener.java f928476 > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserServer.java 877bc08 > exec/java-exec/src/main/java/org/apache/drill/exec/rpc/user/UserSession.java 19d77b0 > exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java dbf3c74 > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/FallbackOptionManager.java 4e90616 > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionManager.java 0b8811a > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/OptionValidator.java 43071e7 > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SessionOptionManager.java c3de190 > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java 4471d4f > exec/java-exec/src/main/java/org/apache/drill/exec/server/options/TypeValidators.java b9721cc > exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java 62f5bdb > exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjection.java 68cbf08 > exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExceptionInjector.java 54bc351 > exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControls.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/Injection.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionConfigurationException.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/InjectionSite.java 9e19fdd > exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/PauseInjection.java PRE-CREATION > exec/java-exec/src/main/java/org/apache/drill/exec/testing/SimulatedExceptions.java 0292c08 > exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java f824b53 > exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java a4a97c9 > exec/java-exec/src/main/java/org/apache/drill/exec/work/user/UserWorker.java 854f474 > exec/java-exec/src/test/java/org/apache/drill/BaseTestQuery.java 3931039 > exec/java-exec/src/test/java/org/apache/drill/PlanningBase.java 264123f > exec/java-exec/src/test/java/org/apache/drill/SingleRowListener.java 99aa9fc > exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestClassTransformation.java f5f5b8d > exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java 7aee6d3 > exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/ParquetResultListener.java 3a794a9 > exec/java-exec/src/test/java/org/apache/drill/exec/store/parquet/TestParquetPhysicalPlan.java cfe52c2 > exec/java-exec/src/test/java/org/apache/drill/exec/testing/ControlsInjectionUtil.java PRE-CREATION > exec/java-exec/src/test/java/org/apache/drill/exec/testing/ExceptionInjectionUtil.java bf93dee > exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestExceptionInjection.java d0c0279 > exec/java-exec/src/test/java/org/apache/drill/exec/testing/TestPauseInjection.java PRE-CREATION > exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillResultSetImpl.java d2302fb > protocol/readme.txt bd516d3 > > Diff: https://reviews.apache.org/r/32795/diff/ > > > Testing > ------- > > Unit tests in TestDrillbitResilience, TestExceptionInjection and TestPauseInjection. > Successful Jenkins builds. > > > Thanks, > > Sudheesh Katkam > > --===============3387930206556218284==--