reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-877) Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs
Date Tue, 17 May 2016 01:28:12 GMT

    [ https://issues.apache.org/jira/browse/REEF-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15285818#comment-15285818
] 

Julia commented on REEF-877:
----------------------------

Whatever we set for the BatchId, will be returned in the Allocated Evaluator so that we know
which one is requested. What is not safe? This one won't be changed/rounded, right? 

> Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs
> --------------------------------------------------------------------------
>
>                 Key: REEF-877
>                 URL: https://issues.apache.org/jira/browse/REEF-877
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF.NET Driver
>            Reporter: Andrew Chung
>            Assignee: Andrew Chung
>
> In the method, we specifically check that the memory of the request matches the allocated
evaluator before removing it from EvaluatorRequestor.Evaluators. That means that if it never
matches, the Evaluator stays there forever and an the AllocatedEvaluatorHandler is never called
with an AllocatedEvaluator that matches the EvaluatorBatchID. This is dangerous because resource
managers such as YARN, for example, can round the memory up for the AllocatedEvaluator. It
is thus likely that if the user doesn't know what memory multiple to round it to, the user
will never get an AllocatedEvaluator matching the EvaluatorBatchID, and EvaluatorRequestor.
Evaluators can fill up with pending requests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message