flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3755) Introduce key groups for key-value state to support dynamic scaling
Date Thu, 14 Apr 2016 09:05:25 GMT
Till Rohrmann created FLINK-3755:
------------------------------------

             Summary: Introduce key groups for key-value state to support dynamic scaling
                 Key: FLINK-3755
                 URL: https://issues.apache.org/jira/browse/FLINK-3755
             Project: Flink
          Issue Type: New Feature
    Affects Versions: 1.1.0
            Reporter: Till Rohrmann
            Assignee: Till Rohrmann


In order to support dynamic scaling, it is necessary to sub-partition the key-value states
of each operator. This sub-partitioning, which produces a set of key groups, allows to easily
scale in and out Flink jobs by simply reassigning the different key groups to the new set
of sub tasks. The idea of key groups is described in this design document [1]. 

[1] https://docs.google.com/document/d/1G1OS1z3xEBOrYD4wSu-LuBCyPUWyFd9l3T9WyssQ63w/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message