flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4197) Allow Kinesis Endpoint to be Overridden via Config
Date Tue, 12 Jul 2016 12:59:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372835#comment-15372835
] 

ASF GitHub Bot commented on FLINK-4197:
---------------------------------------

Github user skidder commented on the issue:

    https://github.com/apache/flink/pull/2227
  
    Thanks for the comment, @tzulitai !
    
    I have not had any problems with testing Flink against a local Kinesalite instance where
the Region (required by Flink) is set to `us-east-1` and the endpoint is set to an address
on localhost. The `setRegion` function on the AWS Kinesis client calculates the endpoint URL
from the region name and sets it on the client. The `setEndpoint` function is meant to allow
for testing against non-AWS endpoints, or even new AWS endpoints in Regions unrecognized by
an older version of the library.
    
    I think that requiring the AWS region, and optionally using the AWS endpoint if provided,
will work well; this is consistent with the AWS client's own behavior.


> Allow Kinesis Endpoint to be Overridden via Config
> --------------------------------------------------
>
>                 Key: FLINK-4197
>                 URL: https://issues.apache.org/jira/browse/FLINK-4197
>             Project: Flink
>          Issue Type: Improvement
>          Components: Kinesis Connector
>    Affects Versions: 1.0.3
>            Reporter: Scott Kidder
>            Priority: Minor
>              Labels: easyfix
>             Fix For: 1.0.4
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> I perform local testing of my application stack with Flink configured as a consumer on
a Kinesis stream provided by Kinesalite, an implementation of Kinesis built on LevelDB. This
requires me to override the AWS endpoint to refer to my local Kinesalite server rather than
reference the real AWS endpoint. I'd like to add a configuration property to the Kinesis streaming
connector that allows the AWS endpoint to be specified explicitly.
> This should be a fairly small change and provide a lot of flexibility to people looking
to integrate Flink with Kinesis in a non-production setup.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message