Mailing-List: contact dev-help@reef.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@reef.apache.org
Date: Wed, 18 May 2016 22:45:13 +0000 (UTC)
From: "Julia (JIRA)" <jira@apache.org>
To: dev@reef.apache.org
Message-ID: <JIRA.12945223.1456620583000.229944.1463611513326@Atlassian.JIRA>
In-Reply-To: <JIRA.12945223.1456620583000@Atlassian.JIRA>
References: <JIRA.12945223.1456620583000@Atlassian.JIRA> <JIRA.12945223.1456620583147@arcas>
Subject: [jira] [Commented] (REEF-1223) IMRU Fault Tolerance - restart
 failed evaluators
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 18 May 2016 22:45:15 -0000


    [ https://issues.apache.org/jira/browse/REEF-1223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15290015#comment-15290015 ] 

Julia commented on REEF-1223:
-----------------------------

After some discussions with the team, we think we shouldn't keep two IMRU driver versions. The reasons are:
* IMRU driver with fault tolerant should be be backward compatible in the sense that, if no failure happens, it should behavior as before. 
* Adding fault tolerant is just adding/enhance feature, we should just test and use the code. If find any bugs, fix it. 
* We need to test the code on the cluster before checking in.
* Adding another set of classes would make maintenance hard and code base mess. 

> IMRU Fault Tolerance - restart failed evaluators
> ------------------------------------------------
>
>                 Key: REEF-1223
>                 URL: https://issues.apache.org/jira/browse/REEF-1223
>             Project: REEF
>          Issue Type: New Feature
>          Components: IMRU, REEF.NET
>            Reporter: Julia
>            Assignee: Julia
>              Labels: FT
>         Attachments: REEF Fault Tolerant Technical design.docx
>
>
> Currently in .Net Group Communication and IMRU scenario, if one of the Evaluator failed for whatever reason, all the Evaluators will be killed by the driver. 
> There are multiple levels of fault tolerant. The scenario we would like to support in this JIRA is:
> *  When an evaluator failed, this failed evaluator will be killed and other good Evaluators will stay, but all the tasks running on those Evaluators will be stopped. 
> *  A new Evaluator will be requested and started with the original task. 
> *  Same tasks will be resubmitted to the rest the Evaluators
> *  The topology of those tasks will be kept in the same group communication as before
> *  The data that have been downloaded in those good Evaluators will stay. 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)