drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3845) PartitionSender doesn't send last batch for receivers that already terminated
Date Thu, 14 Jan 2016 20:28:40 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098789#comment-15098789
] 

ASF GitHub Bot commented on DRILL-3845:
---------------------------------------

Github user amansinha100 commented on a diff in the pull request:

    https://github.com/apache/drill/pull/319#discussion_r49780505
  
    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java
---
    @@ -286,7 +286,7 @@ public void flush(boolean schemaChanged) throws IOException {
           //      sender has acknowledged the terminate request. After sending the last batch,
all further batches are
           //      dropped.
           //   3. Partitioner thread is interrupted due to cancellation of fragment.
    -      final boolean isLastBatch = isLast || terminated || Thread.currentThread().isInterrupted();
    +      final boolean isLastBatch = isLast || Thread.currentThread().isInterrupted();
    --- End diff --
    
    If the sender is not sending the "last batch" message, could we be sure that no receiver
is waiting for it ? Like we discussed it seems the MergingReceiver does not behave the same
as UnorderedReceiver in the sense that the MR waits for the last batch. 


> PartitionSender doesn't send last batch for receivers that already terminated
> -----------------------------------------------------------------------------
>
>                 Key: DRILL-3845
>                 URL: https://issues.apache.org/jira/browse/DRILL-3845
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>            Reporter: Deneche A. Hakim
>            Assignee: Jacques Nadeau
>             Fix For: 1.5.0
>
>         Attachments: 29c45a5b-e2b9-72d6-89f2-d49ba88e2939.sys.drill
>
>
> Even if a receiver has finished and informed the corresponding partition sender, the
sender will still try to send a "last batch" to the receiver when it's done. In most cases
this is fine as those batches will be silently dropped by the receiving DataServer, but if
a receiver has finished +10 minutes ago, DataServer will throw an exception as it couldn't
find the corresponding FragmentManager (WorkEventBus has a 10 minutes recentlyFinished cache).
> DRILL-2274 is a reproduction for this case (after the corresponding fix is applied).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message