reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1466) Cancel the blocking message reading and close the task properly
Date Fri, 24 Jun 2016 21:38:16 GMT

    [ https://issues.apache.org/jira/browse/REEF-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348708#comment-15348708
] 

Julia commented on REEF-1466:
-----------------------------

I have sent PR for the change, you can get what are changed from there. 
https://github.com/apache/reef/pull/1052/files

Below are the APIs I have changed so far. Original one has no CancellationTokenSource in the
parameter. If we want to change those for scatter, there will be corresponding APIs that need
to be changed. 

IReduceReceiver
  T Reduce(CancellationTokenSource cancellationSource = null);
IReduceSender
  void Send(T data, CancellationTokenSource cancellationSource = null);
IOperatorTopology
  T ReceiveFromParent(CancellationTokenSource cancellationSource = null);
  T ReceiveFromChildren(IReduceFunction<T> reduceFunction, CancellationTokenSource cancellationSource
= null);

> Cancel the blocking message reading and close the task properly
> ---------------------------------------------------------------
>
>                 Key: REEF-1466
>                 URL: https://issues.apache.org/jira/browse/REEF-1466
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>              Labels: FT
>
> Currently when driver sends an event to kill a task, in IMRU task close handler, we set
a flag for the Call() method to return from next iteration. If the Call() method stuck in
reading messages, we throw IMRUTaskSystemException so that for the IMRU driver to receive
IFailedTask. After the drive killed all the tasks, the drive will resubmit tasks if the system
is recoverable. 
> In REEF-1447, the proposed solution for exceptions throw from task close handler is to
fail the Evaluator. With the current IMRU task close handler, this would make all evaluators
fail after the tasks are killed by driver. If we want to treat the exception in close handler
as FailedEvaluator, we must gracefully handle the task close event instead of throw an exception.

> I would like to re-propose the cancellation token solution we discussed before. Pass
a cancellation token from task all the way to the NodeStruct.GetData(). When the task close
handler is called, cancel the cancellation token for the task to return properly. 
> This will involve some GC/Network method signature changes, some are internal some are
public. 
> Let me know if you have any concerns about this solution. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message