reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boris Shulman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (REEF-936) Support scale-down in REEF
Date Thu, 04 Feb 2016 04:29:39 GMT

    [ https://issues.apache.org/jira/browse/REEF-936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15131718#comment-15131718
] 

Boris Shulman commented on REEF-936:
------------------------------------

My plan is as following:
1) MultiRuntime will provide driver side handlers (RequestHandler, ReleaseHandler, LaunchHandler,
ResourceManagerStartHandler and ResourceManagerStopHandler)that will receive the actual calls
and delegate the calls for the actual runtime.
2) All the handlers above will reference RuntimesContainer that will host the actual runtimes.

3) RuntimesConainer will receive serialized runtime configurations and an injector in its
constructor. It will initialize different runtimes by instantianting different configurations
(different injector for each configuration). We will copy needed objects (RuntimeParameters.ResourceStatusHandler,
RuntimeParameters.NodeDescriptorHandler, RuntimeParameters.ResourceAllocationHandler and RuntimeParameters.RuntimeStatusHandler)
from the original injector to the new injectors.
4) As tang does not support Maps, the RuntimesContainer will receive list of runtime configurations
and a list of runtime names. The first runtime in the list will be used as a default runtime.

WDYT?


> Support scale-down in REEF
> --------------------------
>
>                 Key: REEF-936
>                 URL: https://issues.apache.org/jira/browse/REEF-936
>             Project: REEF
>          Issue Type: Improvement
>          Components: REEF
>            Reporter: Markus Weimer
>            Assignee: Boris Shulman
>
> The minimal useful REEF job right now consists of two containers: One for the Driver
and one for the Evaluator. This is a fitting design for applications that need to scale out.
> However, some apps might need to be elastic to workloads the size of a single Task, at
which point the overhead for the Driver becomes substantial.
> While there is no plan of action yet, let's use this JIRA as the link target for future
work towards this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message