reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergiy Matusevych <sergiy.matusev...@gmail.com>
Subject Re: gRPC based Java Bridge
Date Mon, 16 Apr 2018 23:05:01 GMT
Hi guys,

gRPC sounds great, and the more REEF code we can delete in the process, the
better! For me, however, the most important aspect of this effort is to
create a unified protocol definition for all REEF components. I would love
to be able to go to the REEF code, open a few *.proto files and from that
figure out how the entire system works on the high level. That is, our
.proto files would serve as an executable specification for REEF components.

Cheers,
Sergiy.




On Mon, Apr 16, 2018 at 2:01 PM, Tyson Condie <tcondie.apache@gmail.com>
wrote:

> The key solution that the new bridge will take on involves a dual process
> design, where the core java driver is in one process, and the application
> driver is in the other process. From there, we need to communicate
> information between these two worlds. gRPC+protocol buffers is one way of
> doing that communication. Since protocol buffers are already used between
> driver and evaluator, I felt the same would be best between
> driver-to-driver, and gRPC seems to be a well tested/supported RPC layer
> for protocol buffers.
>
> -Tyson
>
> On Tue, Apr 10, 2018 at 4:08 PM, Byung-Gon Chun <bgchun@gmail.com> wrote:
>
> > Tyson, thanks for the update!
> > Could you give us a background on why a gRPC-based solution's introduced?
> >
> > Thanks!
> > -Gon
> >
> > On Wed, Apr 11, 2018 at 1:59 AM, Tyson Condie <tcondie.apache@gmail.com>
> > wrote:
> >
> >> Hello,
> >>
> >> We (myself, Doug Service and Scott Inglis) are in the process of
> >> developing
> >> a new REEF Java Bridge that will use a two process solution to
> communicate
> >> between the core Java Driver and an application Driver, which could be
> >> implemented in an alternative language e.g., C#.
> >>
> >> Communication between these two worlds will occur over gRPC using
> protocol
> >> buffers 3.5 as the data format. However, the code will be structured in
> a
> >> way that minimizes such dependencies in the case that an
> >> alternative/better
> >> communication medium presents itself.
> >>
> >> Current status: the core Java Driver is nearly code complete and I am
> >> working on an application (client) Driver in Java (as well) that can be
> >> used as a template for the C# application Driver, and for authoring unit
> >> tests. Please expect a pull request with these changes in the coming
> days.
> >> Concurrently, Doug Service and Scott Inglis will be developing a C#
> based
> >> application Driver.
> >>
> >> The core Java Driver and Java client/application Driver changes can be
> >> tracked via Jira 2002 (https://issues.apache.org/jira/browse/REEF-2002
> ),
> >> which is a subtask to Jira 335 for removing the managed C++ Java bridge.
> >>
> >> Your feedback would be most welcome!
> >>
> >> Thanks
> >> Tyson
> >>
> >
> >
> >
> > --
> > Byung-Gon Chun
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message