bookkeeper-distributedlog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sijie Guo <si...@apache.org>
Subject Re: Add DistributedLog IO
Date Fri, 02 Sep 2016 04:41:23 GMT
Wow. This sounds interesting. Look forward to this. Let us know if you need
any helps.

- Sijie

On Wed, Aug 31, 2016 at 2:02 AM, Khurrum Nasim <khurrumnasimm@gmail.com>
wrote:

> Hello beam folks,
>
> We are evaluating a new solution to unify our streaming and batching data
> pipeline, from storage, computing engine to programming model. The idea is
> basically to implement the Kappa architecture, using DistributedLog as a
> unified stream store for both streaming and batching, using Flink or Spark
> (still debating) as the process engine, and using Beam as the programming
> model.
>
> We'd like to contribute an IO connector to DistributedLog (both bounded
> source/sink and unbounded source/sink).
>
> Is there any special instructions or best practise to add a new IO
> connector? Any suggestion is very appreciated.
>
> The jira is here: https://issues.apache.org/jira/browse/BEAM-607
>
> Also, /cc the distributed log team for any helps.
>
> KN
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message