flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3230) Kinesis streaming producer
Date Wed, 13 Jan 2016 10:38:39 GMT
Tzu-Li (Gordon) Tai created FLINK-3230:

             Summary: Kinesis streaming producer
                 Key: FLINK-3230
                 URL: https://issues.apache.org/jira/browse/FLINK-3230
             Project: Flink
          Issue Type: Sub-task
          Components: Streaming Connectors
            Reporter: Tzu-Li (Gordon) Tai

Add a FlinkKinesisProducer for the Flink Kinesis streaming connector. We will be using AWS
SDK implementation for code consistency with the FlinkKinesisConsumer.

The features of FlinkKinesisProducer is rather straightforward:
1. Partition put records based on partition key.
2. Configurable put mode: Bulk put for higher throughput vs. sequential single record puts.
Size of bulk should also be configurable.
3. For bulk put, user can also choose to enforce strict ordering of the result with the tradeoff
of higher put latency. Ref: https://brandur.org/kinesis-order

This message was sent by Atlassian JIRA

View raw message