reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Weimer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-1304) Create tests which use .NET Poison to validate our failure scenarios
Date Wed, 27 Apr 2016 16:35:12 GMT

    [ https://issues.apache.org/jira/browse/REEF-1304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260437#comment-15260437
] 

Markus Weimer commented on REEF-1304:
-------------------------------------

The tests in the table all assume Evaluator failure, right? We should also have tests for
Task failure. I believe [~juliaw] has done such a test recently. It would be good to group
them all in a common name space, e.g. {{O.A.R.Tests.Functional.Failure}} or such. Task failure
can happen between submission and {{RunningTask}} as well as between {{RunningTask}} and {{CompletedTask}}.
An interesting edge case is failure between {{IRunningTask.Dispose()}} and {{CompletedTask}}:
In this case, the Task failed to shut down. In all of these cases, it should be possible to
submit another Task to the still running Evaluator.

> Create tests which use .NET Poison to validate our failure scenarios
> --------------------------------------------------------------------
>
>                 Key: REEF-1304
>                 URL: https://issues.apache.org/jira/browse/REEF-1304
>             Project: REEF
>          Issue Type: Test
>          Components: REEF.NET
>            Reporter: Mariia Mykhailova
>            Assignee: Mariia Mykhailova
>              Labels: FT
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> There are multiple scenarios of failures in our system. We should use REEF.NET Poison
(delayed fault injection mechanism) to simulate these scenarios in test and to validate the
response of our system to them.
> Here is initial list of tests we need to have.
> || N || Failure injected at || Expected events || Additional verification ||
> | 1 | ON_EVALUATOR_ALLOCATED | FailedEvaluator | |
> | 2 | between ON_EVALUATOR_ALLOCATED and ON_CONTEXT_ACTIVE | FailedEvaluator | |
> | 3 | ON_CONTEXT_ACTIVE | FailedEvaluator + FailedContext | FailedEvaluator should have
FailedContexts filled |
> | 4 | between ON_CONTEXT_ACTIVE and ON_TASK_RUNNING | FailedEvaluator + FailedContext
| FailedEvaluator should have FailedContexts filled |
> | 5 | ON_TASK_RUNNING | FailedEvaluator + FailedContext + FailedTask | FailedEvaluator
should have FailedContexts and FailedTask filled |
> | 6 | between ON_TASK_RUNNING and ON_TASK_COMPLETED | FailedEvaluator + FailedContext
+ FailedTask | FailedEvaluator should have FailedContexts and FailedTask filled |
> | 7 | ON_TASK_COMPLETED | FailedEvaluator + FailedContext | FailedEvaluator should have
FailedContexts filled |
> | 8 | after ON_TASK_COMPLETED | FailedEvaluator ? | race against evaluator completion
|



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message