flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harikrishnan S <hihari...@gmail.com>
Subject Re: JDBC sink in flink
Date Tue, 05 Jul 2016 10:51:49 GMT
Ah that makes send. Also what's the difference between a RichOutputFormat
and a RichSinkFunction ? Can I use JDBCOutputFormat as a sink in a stream ?

On Tue, Jul 5, 2016 at 3:53 PM, Chesnay Schepler <chesnay@apache.org> wrote:

> Hello,
>
> an instance of the JDBCOutputFormat will use a single connection to send
> all values.
>
> Essentially
> - open(...) is called at the very start to create the connection
> - then all invoke/writeRecord calls are executed (using the same
> connection)
> - then close() is called to clean up.
>
> The total number of connections made to the database depends on the
> parallelism of the Sink, as every parallel instance creates it's own
> connection.
>
> Regards,
> Chesnay
>
>
> On 05.07.2016 12:04, Harikrishnan S wrote:
>
> The basic idea was that I would create a pool of connections in the open()
> method in a custom sink and each invoke() method gets one connection from
> the pool and does the upserts needed. I might have misunderstood how sinks
> work in flink though.
>
> On Tue, Jul 5, 2016 at 2:22 PM, Flavio Pompermaier <pompermaier@okkam.it>
> wrote:
>
>> why do you need a connection pool?
>> On 5 Jul 2016 11:41, "Harikrishnan S" <hihari777@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Are there any examples of implementing a jdbc sink in flink using a
>>> connection pool ?
>>>
>>> Thanks
>>>
>>> On Tue, Jul 5, 2016 at 2:00 PM, Harikrishnan S < <hihari777@gmail.com>
>>> hihari777@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Are there any examples of implementing a jdbc sink in flink using a
>>>> connection pool ?
>>>>
>>>> Thanks
>>>>
>>>> On Tue, Jul 5, 2016 at 1:57 PM, Harikrishnan S < <hihari777@gmail.com>
>>>> hihari777@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Are there any examples of implementing a jdbc sink in flink using a
>>>>> connection pool ?
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>>
>
>

Mime
View raw message