flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: JDBC Streaming Connector
Date Fri, 26 Feb 2016 09:33:52 GMT
Hi Timothy!

I would first just write a stream data sink based on the JDBCOutputFormat,
and then let's look into making it exactly-once.

For JDBC with support for transactions, the basic idea would be to start a
transaction, insert elements, commit when a checkpoint is confirmed, and
start a new transaction.

On Fri, Feb 26, 2016 at 10:28 AM, Till Rohrmann <trohrmann@apache.org>
wrote:

> Hi Timothy,
>
> you can use the `JDBCOutputFormat` in combination with the
> `DataStream.writeUsingOutputFormat` method. However, this won't give you
> exactly once guarantees since the output formats don't take part in the
> checkpointing mechanism.
>
> Currently, Chesnay is working on a generic framework for almost exactly
> once sinks. Maybe you can sync with him. You'll find the corresponding JIRA
> ticket here [1].
>
> [1] https://issues.apache.org/jira/browse/FLINK-3332
>
> Cheers,
> Till
>
>
> On Fri, Feb 26, 2016 at 1:23 AM, Timothy Farkas <
> timothytiborfarkas@gmail.com> wrote:
>
> > Hi All,
> >
> > I'm new to Flink and was looking for a JDBC stream sink connector, and
> > didn't see one in flink-streaming-connectors. Is there one somewhere
> else,
> > or is there one currently in development. If not could I pick up a ticket
> > to add it?
> >
> > Thanks,
> > Tim
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message