reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingda Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-6) Improve allocation performance
Date Tue, 28 Oct 2014 17:51:35 GMT

    [ https://issues.apache.org/jira/browse/REEF-6?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14187175#comment-14187175
] 

Yingda Chen commented on REEF-6:
--------------------------------

John,

This does not seem to work with our case, even for the simpler ones. I tried 

1) <1, 512MB> <1, 1999MB>
2) <1, 1024MB> <1, 2048MB>

Both hang without any Evaluator launched (there was some container allocation, but it never
made it through YARNContainerManger to be translated into REEF Evaluator). 

Please note that in our case, all racks are enforced to be "*", you may try that to see if
you can get a repro by putting "*" as rack description.

 

> Improve allocation performance
> ------------------------------
>
>                 Key: REEF-6
>                 URL: https://issues.apache.org/jira/browse/REEF-6
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF-Runtime-YARN
>            Reporter: Markus Weimer
>            Assignee: Yingda Chen
>            Priority: Critical
>
> REEF today uses a very conservative protocol when interacting with YARN, which only allows
for one allocation request to be in flight at a time. 
> We can improve upon this by allowing one kind of allocation request to be in flight at
a time without giving up the safety of the current approach.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message