reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Yang <johnya...@gmail.com>
Subject Re: Ability to support multiple Tasks on an Evaluator
Date Fri, 12 Feb 2016 08:33:00 GMT
Thanks Andrew for the explanation, the plan sounds good.
Yes, using the same names can be a bit confusing, however I'm afraid I
don't have any good ideas either.

Thanks,
John


On Fri, Feb 12, 2016 at 4:51 PM, Andrew Chung <afchung90@gmail.com> wrote:

> Thanks for the feedback John!
> Yes, I believe Vortex can certainly benefit from the service, although
> it will likely be implemented in C# first.
> I was thinking about simply using Task#sendMessage, since the host
> Task will be the one to spin off additional Tasks on the Evaluator,
> just like in Vortex. Sorry for the overloading of terms, I don't have
> a good name for these sub-tasks. Any suggestions for the name?
>
> Thanks,
> Andrew
>
> On Wed, Feb 10, 2016 at 10:47 PM, John Yang <johnyangk@gmail.com> wrote:
> > Hi,
> >
> >
> > Implementing the Multi-Task API using a service sounds great.  We can
> even
> > re-write Vortex to use the API, doing without the boilerplate code.
> >
> > I think the following parameters used in Vortex can also be included in
> the
> > Multi-Task configuration.
> >
> >    - Maximum number of tasks that can concurrently run at a time within
> an
> >    Evaluator
> >    - Maximum number of tasks that can wait to be executed (equal or
> bigger
> >    than the first parameter) within an Evaluator
> >
> > Also, it'd be great if the user can send custom messages from the Driver
> to
> > an Evaluator. I guess this can be done using an existing channel
> > like ActiveContext#sendMessage?
> >
> >
> > Thanks,
> > John
> >
> >
> > On Fri, Jan 29, 2016 at 7:46 AM, Byung-Gon Chun <bgchun@gmail.com>
> wrote:
> >
> >> On Fri, Jan 29, 2016 at 3:17 AM, Andrew Chung <afchung90@gmail.com>
> wrote:
> >>
> >> > Hi,
> >> >
> >> > Thanks for the comments Markus!
> >> >
> >> > A way I am thinking to implement this now is as follows:
> >> > User calls SubmitTask with a Multi-Task TaskConfiguration. The
> >> > Multi-Task Task starts a service on top of the Evaluator, and accepts
> >> > incoming tasks and injects REEF Tasks based on the TaskConfiguration
> >> > that the user passes to the service: this way, the user doesn't need
> >> > to rewrite their Tasks to fit into a new Multi-Task object. In order
> >> > for tasks to use contexts, we will need to expose a hook from Core
> >> > REEF such that the ContextConfiguration of a REEF Context is bound to
> >> > the user's Multi-Task TaskConfiguration.
> >> >
> >>
> >> With Multi-Task, can you run "Tasks" (it may be good to introduce a new
> >> term, REEF Task is confusing to me) concurrently?
> >> (We may be able to re-use some of the Vortex codebase.)
> >>
> >> Can you share specific use cases of Multi-Task you have in mind?
> >>
> >>
> >>
> >> > I'm sure there are some other issues that I am overlooking right now,
> >> > but they will appear as we begin implementing the service. Our first
> >> > question to answer, however, is whether or not this is a service that
> >> > people would like to have in the open source codebase. What do you
> >> > think?
> >> >
> >>
> >> Yes, of course.
> >>
> >> Thanks!
> >>
> >>
> >> >
> >> > Thanks,
> >> > Andrew
> >> >
> >> > On Wed, Jan 27, 2016 at 4:38 PM, Markus Weimer <markus@weimo.de>
> wrote:
> >> > > On 2016-01-27 15:47, Andrew Chung wrote:
> >> > >>
> >> > >> The request to support multiple Tasks on a single Evaluator has
> been
> >> > >> coming up quite a bit internally within Microsoft. Is this a
> feature
> >> > >> that everyone would like to use? Should this be in core REEF?
Or
> >> > >> should this be built as an application on top of REEF?
> >> > >
> >> > >
> >> > > +1 on treating it as a service like naming service or the group
> >> > > communications service. If possible, we should avoid breakage of
> core
> >> > APIs.
> >> > >
> >> > > That said, if we don't have the needed hooks for a layer like this,
> >> then
> >> > we
> >> > > should by all means add more hooks into core REEF.
> >> > >
> >> > > One question I have is whether or not today's applications need to
> be
> >> > able
> >> > > to execute on this new multi-task layer. If not, this new
> functionality
> >> > can
> >> > > be implemented by a single `Task` on the Evaluator side and
> presented
> >> as
> >> > a
> >> > > bunch of new APIs / Events / Event Handlers on the Driver side.
> >> > >
> >> > > Or am I missing something?
> >> > >
> >> > > Markus
> >> >
> >>
> >>
> >>
> >> --
> >> Byung-Gon Chun
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message