flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Farkas <timothytiborfar...@gmail.com>
Subject Re: JDBC Streaming Connector
Date Fri, 26 Feb 2016 17:39:57 GMT
Thanks everyone. I'll take a look at JDBCOutputFormat, and FLINK-3332
<https://issues.apache.org/jira/browse/FLINK-3332> along with other related
tickets.

Tim

On Fri, Feb 26, 2016 at 1:55 AM, Chesnay Schepler <c.schepler@web.de> wrote:

> I'm actually working on a JDBC streaming connector with transactions.
>
>
> On 26.02.2016 10:33, Stephan Ewen wrote:
>
>> Hi Timothy!
>>
>> I would first just write a stream data sink based on the JDBCOutputFormat,
>> and then let's look into making it exactly-once.
>>
>> For JDBC with support for transactions, the basic idea would be to start a
>> transaction, insert elements, commit when a checkpoint is confirmed, and
>> start a new transaction.
>>
>> On Fri, Feb 26, 2016 at 10:28 AM, Till Rohrmann <trohrmann@apache.org>
>> wrote:
>>
>> Hi Timothy,
>>>
>>> you can use the `JDBCOutputFormat` in combination with the
>>> `DataStream.writeUsingOutputFormat` method. However, this won't give you
>>> exactly once guarantees since the output formats don't take part in the
>>> checkpointing mechanism.
>>>
>>> Currently, Chesnay is working on a generic framework for almost exactly
>>> once sinks. Maybe you can sync with him. You'll find the corresponding
>>> JIRA
>>> ticket here [1].
>>>
>>> [1] https://issues.apache.org/jira/browse/FLINK-3332
>>>
>>> Cheers,
>>> Till
>>>
>>>
>>> On Fri, Feb 26, 2016 at 1:23 AM, Timothy Farkas <
>>> timothytiborfarkas@gmail.com> wrote:
>>>
>>> Hi All,
>>>>
>>>> I'm new to Flink and was looking for a JDBC stream sink connector, and
>>>> didn't see one in flink-streaming-connectors. Is there one somewhere
>>>>
>>> else,
>>>
>>>> or is there one currently in development. If not could I pick up a
>>>> ticket
>>>> to add it?
>>>>
>>>> Thanks,
>>>> Tim
>>>>
>>>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message