flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3755) Introduce key groups for key-value state to support dynamic scaling
Date Wed, 07 Sep 2016 17:40:21 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15471283#comment-15471283

ASF GitHub Bot commented on FLINK-3755:

Github user StefanRRichter commented on the issue:

    I don't think that this is giving us the correct inverse for ```computeOperatorIndexForKeyGroup(...)```.
Our test ```CheckpointCoordinatorTest::testCreateKeyGroupPartitions()```generates counter-examples.

> Introduce key groups for key-value state to support dynamic scaling
> -------------------------------------------------------------------
>                 Key: FLINK-3755
>                 URL: https://issues.apache.org/jira/browse/FLINK-3755
>             Project: Flink
>          Issue Type: New Feature
>    Affects Versions: 1.1.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>             Fix For: 1.2.0
> In order to support dynamic scaling, it is necessary to sub-partition the key-value states
of each operator. This sub-partitioning, which produces a set of key groups, allows to easily
scale in and out Flink jobs by simply reassigning the different key groups to the new set
of sub tasks. The idea of key groups is described in this design document [1]. 
> [1] https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing

This message was sent by Atlassian JIRA

View raw message