reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Douglas Service <dsop...@gmail.com>
Subject Re: REEF runs from Spark!
Date Thu, 27 Apr 2017 21:17:47 GMT
I believe this needs to be tested on a spark cluster in addition to normal
tests? If so can  I use an HDInsight Ubuntu Spark cluster?

On Thu, Apr 27, 2017 at 1:36 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:

> Thank you guys! Now please someone merge
> https://github.com/apache/reef/pull/1299 :-)
>
> On Thu, Apr 27, 2017 at 1:13 PM, Douglas Service <dsopsrc@gmail.com>
> wrote:
>
> > Congratulations Sergiy!
> >
> > On Thu, Apr 27, 2017 at 6:52 PM, Markus Weimer <markus@weimo.de> wrote:
> >
> > > Fantastic news! Thanks so much for this!
> > >
> > > Next, we should make this one of our examples :)
> > >
> > > Markus
> > >
> > > On Thu, Apr 27, 2017 at 10:13 AM, Rogan Carr <rogan.carr@gmail.com>
> > wrote:
> > > > Nice work Sergiy!
> > > >
> > > > Best,
> > > > Rogan
> > > >
> > > > On Wed, Apr 26, 2017 at 10:55 PM, Sergiy Matusevych <
> > > > sergiy.matusevych@gmail.com> wrote:
> > > >
> > > >> Thank you Gon!
> > > >> Now we need to fix the remaining issues and we'll have an awesome
> 0.16
> > > >> release. Lean in, reef devs! :)
> > > >>
> > > >> -- Sergiy.
> > > >>
> > > >> On Wed, Apr 26, 2017 at 10:32 PM, Byung-Gon Chun <bgchun@gmail.com>
> > > wrote:
> > > >>
> > > >> > Great job!
> > > >> > This is awesome! Very exciting!
> > > >> >
> > > >> > -Gon
> > > >> >
> > > >> > On Thu, Apr 27, 2017 at 2:28 PM, Sergiy Matusevych <
> > > >> > sergiy.matusevych@gmail.com> wrote:
> > > >> >
> > > >> > > Hi fellow reefers,
> > > >> > >
> > > >> > > After almost a year of refactorings and bugfixes, I was
able to
> > run
> > > >> REEF
> > > >> > > application from Spark today. Woo-hoo! All I tried so far
was
> > Hello
> > > >> REEF
> > > >> > > application running from a Spark driver on YARN, but other
> > > >> Java/REEF/YARN
> > > >> > > applications should also run from Spark just fine. I'll
keep
> > > testing it
> > > >> > > tomorrow.
> > > >> > >
> > > >> > > There is a PR pending: https://github.com/apache/reef/pull/1299
> > > >> > >
> > > >> > > We still have a lot of work to do - we need to try (and
write
> unit
> > > >> > tests!)
> > > >> > > for various failure scenarios, test on different versions
of
> YARN,
> > > fix
> > > >> > > outstanding related bugs, such as
> > > >> > > https://issues.apache.org/jira/browse/REEF-1782 , and, most
> > > >> importantly,
> > > >> > > make sure we can launch .NET applications from Spark.
> > > >> > >
> > > >> > > There is a lot of fun work to do - please check out the
JIRA at
> > > >> > > https://issues.apache.org/jira/browse/REEF and send us your
> pull
> > > >> > requests!
> > > >> > >
> > > >> > > Cheers,
> > > >> > > Sergiy.
> > > >> > >
> > > >> > > P.S. I have a sample Spark+REEF app in a private repo. We
will
> > open
> > > >> > source
> > > >> > > it soon, although there is not much to open source - it's
> > basically
> > > a
> > > >> > > REEF-on-REEF example (namely, a try block from
> > > >> > > org.apache.reef.examples.reefonreef.ReefOnReefDriver),
> > transplanted
> > > >> > almost
> > > >> > > verbatim into the Spark driver.
> > > >> > >
> > > >> >
> > > >> >
> > > >> >
> > > >> > --
> > > >> > Byung-Gon Chun
> > > >> >
> > > >>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message