reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Byung-Gon Chun <>
Subject Re: Ability to support multiple Tasks on an Evaluator
Date Thu, 28 Jan 2016 22:46:25 GMT
On Fri, Jan 29, 2016 at 3:17 AM, Andrew Chung <> wrote:

> Hi,
> Thanks for the comments Markus!
> A way I am thinking to implement this now is as follows:
> User calls SubmitTask with a Multi-Task TaskConfiguration. The
> Multi-Task Task starts a service on top of the Evaluator, and accepts
> incoming tasks and injects REEF Tasks based on the TaskConfiguration
> that the user passes to the service: this way, the user doesn't need
> to rewrite their Tasks to fit into a new Multi-Task object. In order
> for tasks to use contexts, we will need to expose a hook from Core
> REEF such that the ContextConfiguration of a REEF Context is bound to
> the user's Multi-Task TaskConfiguration.

With Multi-Task, can you run "Tasks" (it may be good to introduce a new
term, REEF Task is confusing to me) concurrently?
(We may be able to re-use some of the Vortex codebase.)

Can you share specific use cases of Multi-Task you have in mind?

> I'm sure there are some other issues that I am overlooking right now,
> but they will appear as we begin implementing the service. Our first
> question to answer, however, is whether or not this is a service that
> people would like to have in the open source codebase. What do you
> think?

Yes, of course.


> Thanks,
> Andrew
> On Wed, Jan 27, 2016 at 4:38 PM, Markus Weimer <> wrote:
> > On 2016-01-27 15:47, Andrew Chung wrote:
> >>
> >> The request to support multiple Tasks on a single Evaluator has been
> >> coming up quite a bit internally within Microsoft. Is this a feature
> >> that everyone would like to use? Should this be in core REEF? Or
> >> should this be built as an application on top of REEF?
> >
> >
> > +1 on treating it as a service like naming service or the group
> > communications service. If possible, we should avoid breakage of core
> APIs.
> >
> > That said, if we don't have the needed hooks for a layer like this, then
> we
> > should by all means add more hooks into core REEF.
> >
> > One question I have is whether or not today's applications need to be
> able
> > to execute on this new multi-task layer. If not, this new functionality
> can
> > be implemented by a single `Task` on the Evaluator side and presented as
> a
> > bunch of new APIs / Events / Event Handlers on the Driver side.
> >
> > Or am I missing something?
> >
> > Markus

Byung-Gon Chun

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message