reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingda Chen (JIRA)" <>
Subject [jira] [Commented] (REEF-6) Improve allocation performance
Date Wed, 29 Oct 2014 22:31:33 GMT


Yingda Chen commented on REEF-6:

I did more testing and was able to root cause this: the problem is that we are setting vCores
number in our EvaluatorRequest. Things seem to work just fine if no core number is set.

So just try to set the core number by .setNumberOfCores(#num) when you build the EvaluatorRequest,
#num can be 2 for example, then you should have a repro.

> Improve allocation performance
> ------------------------------
>                 Key: REEF-6
>                 URL:
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF-Runtime-YARN
>            Reporter: Markus Weimer
>            Assignee: Yingda Chen
>            Priority: Critical
>         Attachments: driver.stderr, empty-rack-driver.stderr
> REEF today uses a very conservative protocol when interacting with YARN, which only allows
for one allocation request to be in flight at a time. 
> We can improve upon this by allowing one kind of allocation request to be in flight at
a time without giving up the safety of the current approach.

This message was sent by Atlassian JIRA

View raw message