Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id B4216200B3E for ; Wed, 7 Sep 2016 19:40:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id B2B24160AD5; Wed, 7 Sep 2016 17:40:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 2D8D4160ABF for ; Wed, 7 Sep 2016 19:40:22 +0200 (CEST) Received: (qmail 2299 invoked by uid 500); 7 Sep 2016 17:40:21 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 2262 invoked by uid 99); 7 Sep 2016 17:40:21 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Sep 2016 17:40:21 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 2DEB02C1B85 for ; Wed, 7 Sep 2016 17:40:21 +0000 (UTC) Date: Wed, 7 Sep 2016 17:40:21 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (FLINK-3755) Introduce key groups for key-value state to support dynamic scaling MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 07 Sep 2016 17:40:22 -0000 [ https://issues.apache.org/jira/browse/FLINK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471283#comment-15471283 ] ASF GitHub Bot commented on FLINK-3755: --------------------------------------- Github user StefanRRichter commented on the issue: https://github.com/apache/flink/pull/2440 I don't think that this is giving us the correct inverse for ```computeOperatorIndexForKeyGroup(...)```. Our test ```CheckpointCoordinatorTest::testCreateKeyGroupPartitions()```generates counter-examples. > Introduce key groups for key-value state to support dynamic scaling > ------------------------------------------------------------------- > > Key: FLINK-3755 > URL: https://issues.apache.org/jira/browse/FLINK-3755 > Project: Flink > Issue Type: New Feature > Affects Versions: 1.1.0 > Reporter: Till Rohrmann > Assignee: Till Rohrmann > Fix For: 1.2.0 > > > In order to support dynamic scaling, it is necessary to sub-partition the key-value states of each operator. This sub-partitioning, which produces a set of key groups, allows to easily scale in and out Flink jobs by simply reassigning the different key groups to the new set of sub tasks. The idea of key groups is described in this design document [1]. > [1] https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing -- This message was sent by Atlassian JIRA (v6.3.4#6332)