flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Weise (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-8516) FlinkKinesisConsumer does not balance shards over subtasks
Date Fri, 26 Jan 2018 04:27:03 GMT
Thomas Weise created FLINK-8516:
-----------------------------------

             Summary: FlinkKinesisConsumer does not balance shards over subtasks
                 Key: FLINK-8516
                 URL: https://issues.apache.org/jira/browse/FLINK-8516
             Project: Flink
          Issue Type: Bug
          Components: Kinesis Connector
            Reporter: Thomas Weise


The hash code of the shard is used to distribute discovered shards over subtasks round robin.
This works as long as shard identifiers are sequential. After shards are rebalanced in Kinesis,
that may no longer be the case and the distribution become skewed.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message