reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Byung-Gon Chun <bgc...@gmail.com>
Subject Re: new runtime: stand-alone distributed runtime?
Date Fri, 25 Dec 2015 07:22:56 GMT
Thanks for the comments, Boris and Markus.

Boris, the dev/test case is a good use case for the distributed runtime. I
don't think this runtime's for production for the typical use cases we've
been thinking about.
But there are also users who would like to use the reef abstraction in the
environment without resource managers. E.g., running experimental jobs in a
small cluster, running jobs in mobile devices.

Markus, can you elaborate the SSH story?

Thanks.
-Gon


On Fri, Dec 25, 2015 at 4:04 PM, Markus Weimer <markus@weimo.de> wrote:

> How would that new runtime distribute things? Via SSH? Or assume that
> a REEF-agent is running on each machine? The latter seems undesirable,
> as I am not convinced we can do a better job than YARN and Mesos for
> that. The former is attractive. Maybe we can even evolve it such that
> the particular choice of remoting technique (SSH or otherwise) is
> abstracted?
>
> Markus
>
> On Thu, Dec 24, 2015 at 6:03 PM, Byung-Gon Chun <bgchun@gmail.com> wrote:
> > Hi,
> >
> > It looks like some users would like to run reef without running resource
> > managers like YARN and Mesos. This is similar to the Spark stand-alone
> > mode.
> > What do you think about adding a new stand-alone distributed runtime?
> This
> > runtime can be handy for users who would like to use the reef abstraction
> > to build their applications without being tied to resource managers.
> > Luckily, the SNU team has a few winter interns ;)
> >
> > Thought on the idea?
> >
> > Thanks.
> > -Gon
> >
> > --
> > Byung-Gon Chun
>



-- 
Byung-Gon Chun

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message