reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dhruv Mahajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1404) IMRU task state Maintenance and Preservation in Evaluator for fault tolerant
Date Thu, 22 Sep 2016 18:54:20 GMT

    [ https://issues.apache.org/jira/browse/REEF-1404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15514161#comment-15514161
] 

Dhruv Mahajan commented on REEF-1404:
-------------------------------------

[~MariiaMykhailova] Thanks a lot for this very nice summarization. Ideally, I think 3 is a
better approach, where memory based checkpointing (by calling Update State) is still done
by the user from within {{UpdateFunction}}. I still believe that this part should be left
down to the developer/user since state update etc. depends on underlying application/algorithm
a lot.

However, when it comes to persisting to remote disk or location it becomes opposite. For example,
in {{UpdateTaskHost}} after broadcast when it is waiting for results from Map function, we
can start writing to the remote location. So in this case giving control to Update task host
makes sense.

So for 3, I am wondering if the interface should be split in two (in memory task maintenance
and persisting to remote location) and then managed at appropriate places.

However, I am perfectly happy to go with 2 for now as first version.

> IMRU task state Maintenance and Preservation in Evaluator for fault tolerant
> ----------------------------------------------------------------------------
>
>                 Key: REEF-1404
>                 URL: https://issues.apache.org/jira/browse/REEF-1404
>             Project: REEF
>          Issue Type: Task
>            Reporter: Julia
>              Labels: FT
>
> IMRU task should be able to 
> * Maintenance and preservation the state
> * When restart, able to recover from the previous sate



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message