reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Chung (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (REEF-1335) Create State Machine for IMRU fault tolerance
Date Thu, 21 Apr 2016 23:48:12 GMT

    [ https://issues.apache.org/jira/browse/REEF-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253015#comment-15253015
] 

Andrew Chung edited comment on REEF-1335 at 4/21/16 11:47 PM:
--------------------------------------------------------------

Thanks for this! I took a brief look at the code and noticed this comment:

{code}
// NB: REEF only uses a single TaskMessage dispatch thread per Evaluator, so TaskMessage handlers
// should never block!
{code}

Taking a look into REEF code, it seems that this is actually configurable via {{DriverConfiguration.EVALUATOR_DISPATCHER_THREADS}},
but it defaults to 1 thread, which begs the question: is this a reasonable default?


was (Author: afchung90):
Thanks for this! I took a brief look at the code and noticed this comment:

{code}
// NB: REEF only uses a single TaskMessage dispatch thread per Evaluator, so TaskMessage handlers
// should never block!
{code}

Taking a look into REEF code, it seems that this is actually configurable via {{DriverCOnfiguration.EVALUATOR_DISPATCHER_THREADS}},
but it defaults to 1 thread, which begs the question: is this a reasonable default?

> Create State Machine for IMRU fault tolerance
> ---------------------------------------------
>
>                 Key: REEF-1335
>                 URL: https://issues.apache.org/jira/browse/REEF-1335
>             Project: REEF
>          Issue Type: Task
>          Components: IMRU, REEF.NET
>            Reporter: Julia
>            Assignee: Julia
>              Labels: FT
>         Attachments: REEF Fault Tolerant Technical design.docx
>
>
> To Support fault tolerant, we would like to use state machine to control the system state
transitions.  
> After driver is created, it will start from request evaluators and submit contexts state;
after all the contexts are ready, it will move to submitting tasks state; when all the tasks
are start running, it moves to tasks running state; when all the tasks are completed, the
state will be changed to tasks completed. If either tasks or evaluators fail, it will change
to shut down state, etc. 
> Here are the proposed system states:
> * WaitingForEvaluator,
> * SubmitingTasks,
> * TasksRunning,
> * TasksCompleted,
> * ShutingDown,
> * Fail
> Here are the event that may trigger the state change:
> * AllContextsAreReady,
> * AllTasksAreRunning,
> * AllTasksAreCompleted,
> * FailedTask,
> * FailedEvaluator,
> * NotRecoverable,
> * Recover



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message