flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tzulitai <...@git.apache.org>
Subject [GitHub] flink pull request #2229: [hotfix][docs] Add note about Kinesis producer lim...
Date Tue, 12 Jul 2016 10:22:47 GMT
Github user tzulitai commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2229#discussion_r70413125
  
    --- Diff: docs/apis/streaming/connectors/kinesis.md ---
    @@ -240,8 +239,9 @@ consumer when calling this API can also be modified by using the other
keys pref
     ### Kinesis Producer
     
     The `FlinkKinesisProducer` is used for putting data from a Flink stream into a Kinesis
stream. Note that the producer is not participating in
    -Flink's checkpointing and doesn't provide exactly-once processing guarantees. In case
of a failure, data will be written again
    -to Kinesis, leading to duplicates. This behavior is usually called "at-least-once" semantics.
    +Flink's checkpointing and doesn't provide exactly-once processing guarantees. 
    +Also, the Kinesis producer does not guarantee that records are written in order to the
shards (See [here](https://github.com/awslabs/amazon-kinesis-producer/issues/23) and [here](http://docs.aws.amazon.com/kinesis/latest/APIReference/API_PutRecord.html#API_PutRecord_RequestSyntax)
for more details). 
    +In case of a failure, data will be written again to Kinesis, leading to duplicates. This
behavior is usually called "at-least-once" semantics.
    --- End diff --
    
    Should we also mention that resharding streams will using the Kinesis Producer will also
result in duplicates? I experienced this while testing exactly-once with resharding before.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message