reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yingda Chen (JIRA)" <>
Subject [jira] [Updated] (REEF-6) Improve allocation performance
Date Wed, 29 Oct 2014 16:29:34 GMT


Yingda Chen updated REEF-6:
    Attachment: driver.stderr


I have attached the driver.stderr for your diagnosis. Unfortunately there is no easy way to
run .NET REEF outside MSFT as we have not yet moved the .NET code to apache. I hope you can
infer something just from the driver.stderr.

Regarding using the asterisk as rack description, we don't have a preference to use it, it
is only there as a result of the unfortunate patch-over-patch to make it "work". Specification
of rack-preference is now allowed on the .net side (although I don't think it is working correctly)
per customer ask, we can certainly disable it for now. Since you are doing an overall of our
evaluator request logic, feel free to make the call on this.

I will try to run it again with empty rack preference.

> Improve allocation performance
> ------------------------------
>                 Key: REEF-6
>                 URL:
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF-Runtime-YARN
>            Reporter: Markus Weimer
>            Assignee: Yingda Chen
>            Priority: Critical
>         Attachments: driver.stderr
> REEF today uses a very conservative protocol when interacting with YARN, which only allows
for one allocation request to be in flight at a time. 
> We can improve upon this by allowing one kind of allocation request to be in flight at
a time without giving up the safety of the current approach.

This message was sent by Atlassian JIRA

View raw message