flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-5590) Create a proper internal state hierarchy
Date Sun, 22 Jan 2017 22:13:27 GMT

     [ https://issues.apache.org/jira/browse/FLINK-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stephan Ewen closed FLINK-5590.
-------------------------------

> Create a proper internal state hierarchy
> ----------------------------------------
>
>                 Key: FLINK-5590
>                 URL: https://issues.apache.org/jira/browse/FLINK-5590
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.2.0
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>             Fix For: 1.3.0
>
>
> Currently, the state interfaces (like {{ListState}}, {{ValueState}}, {{ReducingState}})
are very sparse and contain only methods exposed to the users. That makes sense to keep the
public stable API minimal
> At the same time, the runtime needs more methods for its internal interaction with state,
such as:
>   - setting namespaces
>   - accessing raw values
>   - merging namespaces
> These are currently realized by re-creating or re-obtaining the state objects from the
KeyedStateBackend. That method causes quite an overhead for each access to the state
> The KeyedStateBackend tries to do some tricks to reduce that overhead, but does it only
partially and induces other overhead in the course.
> The root cause of all these issues is a problem in the design: There is no proper "internal
state abstraction" in a similar way as there is an external state abstraction (the public
state API).
> We should add a similar hierarchy of states for the internal methods. It would look like
in the example below:
> {code}
>  *             State
>  *               |
>  *               +-------------------InternalKvState
>  *               |                         |
>  *          MergingState                   |
>  *               |                         |
>  *               +-----------------InternalMergingState
>  *               |                         |
>  *      +--------+------+                  |
>  *      |               |                  |
>  * ReducingState    ListState        +-----+-----------------+
>  *      |               |            |                       |
>  *      +-----------+   +-----------   -----------------InternalListState
>  *                  |                |
>  *                  +---------InternalReducingState
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message