reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergiy Matusevych <sergiy.matusev...@gmail.com>
Subject Re: Write-up about REEF as a library
Date Wed, 15 Feb 2017 21:22:40 GMT
On Wed, Feb 15, 2017 at 1:12 PM, Byung-Gon Chun <bgchun@gmail.com> wrote:


> Could you share information about Spark integration, if it's something you
> can share?
> My students did integrating Spark with REEF a while ago. It'd be
> interesting to compare notes.
>

Hi Gon,

Sure! I am currently trying to make REEF run as a library from the Spark
application master; I have an example on CISL github at:

https://github.com/Microsoft-CISL/reef-spark

(I am not sure if you still have access to it; if you do, you're most
welcome to take a look)

I am currently testing it on our cluster; once we're able to spawn REEF
jobs as part of the Spark pipeline (and then properly clean-up the
resources), I'll go on to implement a small REEF runtime that allows REEF
evaluators run on Spark workers so they can share the resources etc.

P.S. I am intrigued about your students' work. Is this code available
somewhere?

Thank you,
Sergiy.

On Thu, Feb 16, 2017 at 6:03 AM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
> > Hi Gon,
> >
> > Sure, we need to document that work, and I'll be happy to do a write-up.
> > Give me a few weeks to finish my urgent work on Spark integration and I
> > will create a wiki page with code examples. It should not be too hard as
> we
> > already have all code examples in master.
> >
> > Cheers,
> > Sergiy.
> >
> > On Wed, Feb 15, 2017 at 12:55 PM, Byung-Gon Chun <bgchun@gmail.com>
> wrote:
> >
> > > Sergiy, you're the best person to get this started. :)
> > > I'm happy to give feedback on drafts.
> > >
> > >
> > >
> > > On Thu, Feb 16, 2017 at 5:53 AM, Markus Weimer <markus@weimo.de>
> wrote:
> > >
> > > > On 2017-02-15 12:39, Byung-Gon Chun wrote:
> > > >
> > > >> There has been great progress on REEF as a library.
> > > >> It'd be beneficial to have a write-up about it on the REEF website
> for
> > > >> REEF users.
> > > >>
> > > >
> > > > +1
> > > >
> > > > Markus
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message