flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tzulitai <...@git.apache.org>
Subject [GitHub] flink pull request #2227: [FLINK-4197] Allow Kinesis endpoint to be overridd...
Date Tue, 12 Jul 2016 15:15:49 GMT
Github user tzulitai commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2227#discussion_r70459536
  
    --- Diff: docs/apis/streaming/connectors/kinesis.md ---
    @@ -299,3 +299,41 @@ Otherwise, the returned stream name is used.
     Other optional configuration keys can be found in `KinesisConfigConstants`.
     		
     		
    +### Using Non-AWS Kinesis Endpoints for Testing
    +
    +It is sometimes desirable to have Flink operate as a consumer or producer against a non-AWS
Kinesis endpoint such as kinesalite; this is especially useful when performing functional
testing of a Flink application. The AWS endpoint that would normally be inferred by the AWS
region set in the Flink configuration must be overriden via a configuration property.
    +
    +To override the AWS endpoint, set the `KinesisConfigConstants.CONFIG_AWS_ENDPOINT` property
in the Flink configuration, in addition to the `KinesisConfigConstants.CONFIG_AWS_REGION`
required by Flink. Although the region is required, it will not be used to determine the AWS
endpoint URL.
    +
    +The following example shows how one might supply the `KinesisConfigConstants.CONFIG_AWS_ENDPOINT`
configuration property:
    +
    +<div class="codetabs" markdown="1">
    +<div data-lang="java" markdown="1">
    +{% highlight java %}
    +Properties kinesisProducerConfig = new Properties();
    +kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION, "us-east-1");
    +kinesisProducerConfig.put(
    +    KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
    +    "aws_access_key_id_here");
    +kinesisProducerConfig.put(
    +    KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
    +    "aws_secret_key_here");
    +kinesisProducerConfig.put(
    +    KinesisConfigConstants.CONFIG_AWS_ENDPOINT,
    +    "http://localhost:4567");
    +{% endhighlight %}
    +</div>
    +<div data-lang="scala" markdown="1">
    +{% highlight scala %}
    +val kinesisProducerConfig = new Properties();
    +kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION, "us-east-1");
    +kinesisProducerConfig.put(
    +    KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
    +    "aws_access_key_id_here");
    +kinesisProducerConfig.put(
    +    KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
    +    "aws_secret_key_here");
    +kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_ENDPOINT, "http://localhost:4567");
    --- End diff --
    
    nit: This config is written as a single line in the Scala example, but spans 3 lines in
the Java example. Would be good to keep them align :)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message