reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Chung (JIRA)" <>
Subject [jira] [Commented] (REEF-877) Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs
Date Fri, 13 May 2016 00:12:12 GMT


Andrew Chung commented on REEF-877:

[~markus.weimer] it seems like we are actually using this in IMRU. I'll see if we can do anything
for the rounding issue.

> Consider rounding issues in ProcessNewEvaluator() in AllocatedEvaluator.cs
> --------------------------------------------------------------------------
>                 Key: REEF-877
>                 URL:
>             Project: REEF
>          Issue Type: Bug
>          Components: REEF.NET Driver
>            Reporter: Andrew Chung
> In the method, we specifically check that the memory of the request matches the allocated
evaluator before removing it from EvaluatorRequestor.Evaluators. That means that if it never
matches, the Evaluator stays there forever and an the AllocatedEvaluatorHandler is never called
with an AllocatedEvaluator that matches the EvaluatorBatchID. This is dangerous because resource
managers such as YARN, for example, can round the memory up for the AllocatedEvaluator. It
is thus likely that if the user doesn't know what memory multiple to round it to, the user
will never get an AllocatedEvaluator matching the EvaluatorBatchID, and EvaluatorRequestor.
Evaluators can fill up with pending requests.

This message was sent by Atlassian JIRA

View raw message