From issues-return-153281-archive-asf-public=cust-asf.ponee.io@flink.apache.org Thu Feb 15 19:24:07 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 154B718064A for ; Thu, 15 Feb 2018 19:24:06 +0100 (CET) Received: (qmail 5151 invoked by uid 500); 15 Feb 2018 18:24:06 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 5142 invoked by uid 99); 15 Feb 2018 18:24:06 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 15 Feb 2018 18:24:06 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id AAA72C1231 for ; Thu, 15 Feb 2018 18:24:05 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id BjTC1PreS-Rk for ; Thu, 15 Feb 2018 18:24:05 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 7F0905F36A for ; Thu, 15 Feb 2018 18:24:04 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 9D770E030E for ; Thu, 15 Feb 2018 18:24:03 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6670A21E64 for ; Thu, 15 Feb 2018 18:24:01 +0000 (UTC) Date: Thu, 15 Feb 2018 18:24:01 +0000 (UTC) From: "Tzu-Li (Gordon) Tai (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (FLINK-8516) FlinkKinesisConsumer does not balance shards over subtasks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/FLINK-8516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tzu-Li (Gordon) Tai updated FLINK-8516: --------------------------------------- Issue Type: New Feature (was: Bug) > FlinkKinesisConsumer does not balance shards over subtasks > ---------------------------------------------------------- > > Key: FLINK-8516 > URL: https://issues.apache.org/jira/browse/FLINK-8516 > Project: Flink > Issue Type: New Feature > Components: Kinesis Connector > Reporter: Thomas Weise > Assignee: Thomas Weise > Priority: Major > Fix For: 1.5.0 > > > The hash code of the shard is used to distribute discovered shards over subtasks round robin. This works as long as shard identifiers are sequential. After shards are rebalanced in Kinesis, that may no longer be the case and the distribution become skewed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)