flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-8516) Allow custom shard-to-subtask assignment for the FlinkKinesisConsumer
Date Thu, 15 Feb 2018 18:25:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tzu-Li (Gordon) Tai updated FLINK-8516:
---------------------------------------
    Summary: Allow custom shard-to-subtask assignment for the FlinkKinesisConsumer  (was:
FlinkKinesisConsumer does not balance shards over subtasks)

> Allow custom shard-to-subtask assignment for the FlinkKinesisConsumer
> ---------------------------------------------------------------------
>
>                 Key: FLINK-8516
>                 URL: https://issues.apache.org/jira/browse/FLINK-8516
>             Project: Flink
>          Issue Type: New Feature
>          Components: Kinesis Connector
>            Reporter: Thomas Weise
>            Assignee: Thomas Weise
>            Priority: Major
>             Fix For: 1.5.0
>
>
> The hash code of the shard is used to distribute discovered shards over subtasks round
robin. This works as long as shard identifiers are sequential. After shards are rebalanced
in Kinesis, that may no longer be the case and the distribution become skewed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message