streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sblackmon <sblack...@apache.org>
Subject [DISCUSS] Beam
Date Mon, 21 Nov 2016 17:36:24 GMT
Beam appears to be on it’s way to being the de-facto standard for data pipelines.

I’d like to start a real discussion about whether and how to align streams interfaces with
Beam interfaces.

To pose a straw-man theory for discussion:

Hypothesis: Streams would benefit by replacing the interfaces in streams-core entirely with
beam interfaces.
  
a) Do we agree that the flexibility and performance gains from doing so, presuming it’s
possible, would be significant?
b) Are there any inevitable flexiblility, performance, complexity, or other, blockers or compromises
we should discuss?
c) What arguments are there for retaining our interfaces and providing beam compatibility
in a runtime module binding (within streams) vs deprecating our existing interfaces and switching
over completely?
d) Obviously doing this would be a lot of work.  What level of commitment is there from the
group to work on this?

Steve
On October 25, 2016 at 3:47:11 PM, sblackmon (sblackmon@apache.org) wrote:

Regarding Beam, there have been a number of ideas and theories floated on the list and but
nothing concrete has been proposed or discussed in depth.

Steve
On October 25, 2016 at 10:21:52 AM, Suneel Marthi (suneel.marthi@gmail.com) wrote:

Is support for Kafka Streams and Apache Beam on the roadmap ?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message