flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aljoscha Krettek (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (FLINK-3200) Use Partitioned State Abstraction in WindowOperator
Date Wed, 17 Feb 2016 10:04:18 GMT

     [ https://issues.apache.org/jira/browse/FLINK-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Aljoscha Krettek resolved FLINK-3200.
       Resolution: Fixed
    Fix Version/s: 1.0.0

> Use Partitioned State Abstraction in WindowOperator
> ---------------------------------------------------
>                 Key: FLINK-3200
>                 URL: https://issues.apache.org/jira/browse/FLINK-3200
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>    Affects Versions: 1.0.0
>            Reporter: Aljoscha Krettek
>            Assignee: Aljoscha Krettek
>             Fix For: 1.0.0
> Right now, the WindowOperator uses snapshotState and restoreState to do custom serialization/deserialization
of the windowing state. Moving this to partitioned state would allow us to re-scale in the
future. Also, once the partitioned state has support to run on ManagedMemory/Out-of-core state
the WindowOperator would also automatically benefit from this, allowing very large windows
and large window state.
> The necessary steps are these:
> - Enhance state interface with new primitive states: ReducableState, ListState, etc..
Also make state scoped to a namespace. Namespaces are necessary because we need state that
is scoped by key and by window (window would be the namespace in this case)
> - Enhance timer interface to also make timers scoped to a key/namespace
> - Change WindowOperator to use these new interfaces

This message was sent by Atlassian JIRA

View raw message