reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruv Mahajan (JIRA)" <>
Subject [jira] [Commented] (REEF-1392) Adding IObserver<ICloseEvent> for IMRU tasks
Date Thu, 19 May 2016 05:50:12 GMT


Dhruv Mahajan commented on REEF-1392:

So here is I think how it will work:

We will update API of *Function to introduce a function allowing them to checkpoint. This
function will also take as input iteration number. The *Host functions will manage the deletion
of previous checkpoints. At max they need to keep one previous checkpoint since at the time
of failure map and update tasks can have a difference of one iteration count between them.
At the time of restart, update task host can begin by communicating map task host the iteration
number and map task can start from there. However, what about the evaluator that was killed
or taken away. Since it will nor preserve its state at failure, we need to start from the
last periodic checkpoint. So my question is: What is the point of notifying the *Functions
about shutting down except some graceful de-allocations/dispose.

> Adding IObserver<ICloseEvent> for IMRU tasks
> --------------------------------------------
>                 Key: REEF-1392
>                 URL:
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>            Assignee: Dhruv Mahajan
>              Labels: FT
> For fault tolerant, IMRU tasks, MapTaskHost and UpdateTaskHost should implement  IObserver<ICloseEvent>.
When they receive ICloseEvent,  it will verify if the closing event is send from driver based
on the message in the event, then trow IMRUTaskException with a define message to inform the
driver it is closed. 
> The  change should be backward compatible. If the the IMRU tasks are not bound to the
task configuration for  TaskConfiguration.OnClose, the event won't be received. 

This message was sent by Atlassian JIRA

View raw message