flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9124) Allow customization of KinesisProxy.getRecords read timeout and retry
Date Thu, 12 Apr 2018 05:23:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434948#comment-16434948

ASF GitHub Bot commented on FLINK-9124:

Github user tweise commented on a diff in the pull request:

    --- Diff: flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/proxy/KinesisProxy.java
    @@ -176,6 +179,16 @@ protected KinesisProxy(Properties configProps) {
    +	/**
    +	 * Create the Kinesis client, using the provided configuration properties and default
{@link ClientConfiguration}.
    +	 * Derived classes can override this method to customize the client configuration.
    +	 * @param configProps
    +	 * @return
    +	 */
    +	protected AmazonKinesis createKinesisClient(Properties configProps) {
    --- End diff --
    Although it is theoretically possible to override the method and not look at `configProps`,
it is  rather unlikely that this would be unintended. The user that ends up working at this
level will probably be in need to control how the client config is initialized and the client
    is constructed, to make the connector work. My vote is strongly in favor of not locking
down things unless they are extremely well understood and there is a specific reason.
    The connectors in general are fluent by nature and warrant a more flexible approach that

    empowers users to customize what they need without wholesale forking. By now we have run
into several cases where behavior of the Kinesis connector had to be amended but private constructors
or methods got into the way. Who would not prefer to spend time improving the connector functionality
vs. opening JIRAs and PRs for access modification changes?
    In our internal custom code we currently have an override that can generically set any
simple property on the client config from the config properties. The approach comes with its
own pros and cons and I think it should be discussed separately. If there is interest in having
it in the Flink codebase as default behavior, I'm happy to take it up as a separate PR. I
would still want to have the ability to override it though.

> Allow customization of KinesisProxy.getRecords read timeout and retry
> ---------------------------------------------------------------------
>                 Key: FLINK-9124
>                 URL: https://issues.apache.org/jira/browse/FLINK-9124
>             Project: Flink
>          Issue Type: Task
>          Components: Kinesis Connector
>    Affects Versions: 1.4.2
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>            Priority: Minor
> It should be possible to change the socket read timeout and all other configuration parameters
of the underlying AWS ClientConfiguration and also have the option to retry after a socket
timeout exception.

This message was sent by Atlassian JIRA

View raw message