reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariia Mykhailova (JIRA)" <j...@apache.org>
Subject [jira] [Created] (REEF-1677) IMRU: Evaluators failed during WaitingForEvaluator phase don't count towards MaximumNumberOfEvaluatorFailures limit
Date Thu, 24 Nov 2016 01:09:58 GMT
Mariia Mykhailova created REEF-1677:
---------------------------------------

             Summary: IMRU: Evaluators failed during WaitingForEvaluator phase don't count
towards MaximumNumberOfEvaluatorFailures limit
                 Key: REEF-1677
                 URL: https://issues.apache.org/jira/browse/REEF-1677
             Project: REEF
          Issue Type: Bug
          Components: IMRU
            Reporter: Mariia Mykhailova
            Assignee: Mariia Mykhailova


In {{IMRUDriver}}, we keep track of the list of failed evaluator ids, and use its length when
we need to find out how many evaluators failed. However, if evaluator failed in {{WaitingForEvaluator}}
state we immediately remove it from this list and request another evaluator, effectively forgetting
about the failure. Thus, even with lots of evaluators failing at this stage we'll never hit
{{MaximumNumberOfEvaluatorFailures}} limit, and will keep requesting new evaluators indefinitely.

I think we should just remove this list (with the additional benefit of reduced memory consumption)
and replace it with a single counter which is never decremented. We're only using the values
in the list for sanity checks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message