drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti" <venki.koruka...@gmail.com>
Subject Re: Review Request 34014: DRILL-2755: Use and handle InterruptedException during query processing
Date Sat, 09 May 2015 16:41:21 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34014/
-----------------------------------------------------------

(Updated May 9, 2015, 4:41 p.m.)


Review request for drill, Chris Westin and Jacques Nadeau.


Changes
-------

Uploading the actual path


Bugs: DRILL-2755
    https://issues.apache.org/jira/browse/DRILL-2755


Repository: drill-git


Description
-------

DRILL-2755: Use and handle InterruptedException during query processing.

    - Interrupt FragmentExecutor thread as part of FragmentExecutor.cancel()
    - Handle InterruptedException in ExternalSortBatch.newSV2(). If the fragment status says
      should not continue, then throw the InterruptedException to caller which returns IterOutcome.STOP
    - Add comments reg not handling of InterruptedException in SendingAccountor.waitForSendComplete()
    - Handle InterruptedException in OrderedPartitionRecordBatch.getPartitionVectors()
    - If interrupted in Thread.sleep calls and fragment status says should not run, then
      return IterOutcome.STOP downstream.
    - Interrupt partitioner threads if PartitionerRecordBatch is interrupted while waiting
for
      partitioner threads to complete.
    - Preserve interrupt status if not handled
    - Handle null RecordBatches returned by RawBatchBuffer.getNext() in MergingRecordBatch.buildSchema()
    - Change timeout in Foreman to be proportional to the number of intermediate fragments
sent instead
      of hard coded limit of 90s.
    - Change TimedRunnable to enforce a timeout of 15s per runnable.
      Total timeout is (5s * numOfRunnableTasks) / parallelism.


Diffs (updated)
-----

  common/src/main/java/org/apache/drill/common/concurrent/ExtendedLatch.java a75ac32 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/SendingAccountor.java 0cb5fbf 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/mergereceiver/MergingRecordBatch.java
ce683cb 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/orderedpartitioner/OrderedPartitionRecordBatch.java
63b7eba 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
c3261dc 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java
cbea267 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/unorderedreceiver/UnorderedReceiverBatch.java
66a2092 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
e88bc67 
  exec/java-exec/src/main/java/org/apache/drill/exec/record/RawFragmentBatchProvider.java
d4dfe96 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicClient.java 72ae130 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/BasicServer.java 3a7032b 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/ReconnectingConnection.java 9948d3e

  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/RemoteConnection.java a72dd32 
  exec/java-exec/src/main/java/org/apache/drill/exec/rpc/data/DataTunnel.java 11f5496 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/Drillbit.java e7a9a3c 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/TimedRunnable.java 0fb778b 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/CountDownLatchInjection.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/CountDownLatchInjectionImpl.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/ExecutionControlsInjector.java
4b1cd0c 
  exec/java-exec/src/main/java/org/apache/drill/exec/testing/NoOpControlsInjector.java 80d9790

  exec/java-exec/src/main/java/org/apache/drill/exec/testing/PauseInjection.java e5f9c9c 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java a3ceb8f 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/SpoolingRawBatchBuffer.java
2a79e42 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/batch/UnlimitedRawBatchBuffer.java
d23655c 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 49d0c94 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/fragment/FragmentExecutor.java ddb828c

  exec/java-exec/src/test/java/org/apache/drill/exec/server/TestDrillbitResilience.java da69e9e

  exec/java-exec/src/test/java/org/apache/drill/exec/store/TestTimedRunnable.java PRE-CREATION

  exec/jdbc/src/main/java/org/apache/drill/jdbc/impl/DrillResultSetImpl.java 8ef2af3 

Diff: https://reviews.apache.org/r/34014/diff/


Testing
-------

- Add unit tests
      * Testing cancelling a query interrupts the query fragments which are currently blocked
      * Testing interrupting the partitioner sender which in turn interrupts its helper threads
      * Testing TimedRunanble enforeces timeout for the whole task list.


Thanks,

Venki Korukanti


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message