flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: Accessing external DB inside RichFlatMap Function
Date Thu, 07 Jul 2016 09:05:41 GMT
Yes, Chesnay is right. You can open and close the connection in 
the open() and close() methods of your RichFlatMapFunction.


> On Jul 7, 2016, at 11:03 AM, Chesnay Schepler <chesnay@apache.org> wrote:
> Couldn't he do the same thing in his RichFlatMap?
> open the db connection in open(), close it in close(), do stuff within these calls.
> On 07.07.2016 10:58, Kostas Kloudas wrote:
>> Hi Simon,
>> If your job reads or writes to a DB, I would suggest to use one of the already existing
Flink sources or sinks,
>> as this allows for efficient connection handling and managing.
>> If you want to write intermediate data to a DB from an operator, then I suppose that
you should implement
>> your own custom operator that opens a DB connection in the open() method and closes
it at close().
>> If you are planning to do so, I think that code of your custom operator would be
the same as the StreamFlatMap,
>> with the addition of the openDBConnection method in the open(), and you should also
override the close() method
>> of the AbstractUdfStreamOperator to 1) call super.close() and 2) close the db connection.
>> Let me know if this works,
>> Kostas
>>> On Jul 7, 2016, at 10:38 AM, simon peyer <simon.peyer@soom-it.ch> wrote:
>>> Hi guys
>>> Is there a easy way to handle external DB connections inside a RichFlatMap Function?
>>> --Thanks Simon

View raw message