flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From François Lacombe (JIRA) <j...@apache.org>
Subject [jira] [Commented] (FLINK-10315) Let JDBCAppendTableSink be built with java.sql.Connection
Date Mon, 24 Sep 2018 10:22:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-10315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16625611#comment-16625611

François Lacombe commented on FLINK-10315:

HI [~yanghua], thank you for this feedback

Here what I think :
1) Should the sink control the connection it is using? If the provided connection is not usable,
then it should throw an exception. The user should check and guarantee the provided connection
is valid to be used at any time. Is this a problem? I wasn't suggesting to only provide existing
connection, but only to be able to do so. Method providing credentials should remain on the
builder API.

2) In programs I run the Flink sink isn't the only component to use a JDBC connection. As
to efficiently manage the resources I use, I really prefer to initiate a single connection
which will last longer than many connections at run time. Providing sensitive credentials
to external components is a situation for me also.

I understand the serialization issue, which is real. Can't the connection be registered to
a kind of pool and referenced by both the TM and builders?

Kind regards

> Let JDBCAppendTableSink be built with java.sql.Connection
> ---------------------------------------------------------
>                 Key: FLINK-10315
>                 URL: https://issues.apache.org/jira/browse/FLINK-10315
>             Project: Flink
>          Issue Type: Improvement
>          Components: Java API
>         Environment: I'm currently using Flink 1.6.0 Java.
>            Reporter: François Lacombe
>            Assignee: vinoyang
>            Priority: Major
> Currently, JDBCAppendTableSink is built with methods like setDBUrl, setUsername, setPassword...
and so on.
> We can't use an existing Java SQL connection to build it.
> It may be great to add a setConnection() method to the builder class as to prevent sensitive
data like username or password to transit through large stacks from config connectors (often
in main()) to JDBC sinks.
> To be able to provide only one object is far lighter than 4 or 5 strings
> Thanks in advance

This message was sent by Atlassian JIRA

View raw message