drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-6295) PartitionerDecorator may close partitioners while CustomRunnable are active during query cancellation
Date Mon, 16 Apr 2018 19:59:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-6295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16439962#comment-16439962
] 

ASF GitHub Bot commented on DRILL-6295:
---------------------------------------

Github user vrozov commented on a diff in the pull request:

    https://github.com/apache/drill/pull/1208#discussion_r181865979
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
---
    @@ -118,105 +127,114 @@ public PartitionOutgoingBatch getOutgoingBatches(int index) {
         return null;
       }
     
    -  @VisibleForTesting
    -  protected List<Partitioner> getPartitioners() {
    +  List<Partitioner> getPartitioners() {
         return partitioners;
       }
     
       /**
        * Helper to execute the different methods wrapped into same logic
        * @param iface
    -   * @throws IOException
    +   * @throws ExecutionException
        */
    -  protected void executeMethodLogic(final GeneralExecuteIface iface) throws IOException
{
    -    if (partitioners.size() == 1 ) {
    -      // no need for threads
    -      final OperatorStats localStatsSingle = partitioners.get(0).getStats();
    -      localStatsSingle.clear();
    -      localStatsSingle.startProcessing();
    +  @VisibleForTesting
    +  void executeMethodLogic(final GeneralExecuteIface iface) throws ExecutionException
{
    +    // To simulate interruption of main fragment thread and interrupting the partition
threads, create a
    +    // CountDownInject latch. Partitioner threads await on the latch and main fragment
thread counts down or
    +    // interrupts waiting threads. This makes sure that we are actually interrupting
the blocked partitioner threads.
    +    try (CountDownLatchInjection testCountDownLatch = injector.getLatch(context.getExecutionControls(),
"partitioner-sender-latch")) {
    --- End diff --
    
    The `testCountDownLatch` is used only for testing and initialized to 1. The wait is on
`count`. 


> PartitionerDecorator may close partitioners while CustomRunnable are active during query
cancellation
> -----------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-6295
>                 URL: https://issues.apache.org/jira/browse/DRILL-6295
>             Project: Apache Drill
>          Issue Type: Bug
>            Reporter: Vlad Rozov
>            Assignee: Vlad Rozov
>            Priority: Critical
>             Fix For: 1.14.0
>
>
> During query cancellation, in case {{PartitionerDecorator.executeMethodLogic()}} is active
(waiting on the {{latch}}), the wait will be interrupted and {{Futures}} cancelled, but there
is no guarantee that all {{CustomRunnable}} terminate before returning from {{PartitionerDecorator.executeMethodLogic()}}.
On exit, both income and outgoing batches are cleared, leading to clearing of underlying {{Vectors}}
and {{DrillBufs}}. This eventually causes unallocated memory access and JVM crash as {{CustomRunnable}}
may execute after income/outgoing batches are cleared.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message