flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ufuk Celebi (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-3779) Add support for queryable state
Date Tue, 09 Aug 2016 14:51:20 GMT

     [ https://issues.apache.org/jira/browse/FLINK-3779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ufuk Celebi closed FLINK-3779.
------------------------------
       Resolution: Implemented
    Fix Version/s: 1.2.0

Implemented in a909adb~1..490e7eb (master).

> Add support for queryable state
> -------------------------------
>
>                 Key: FLINK-3779
>                 URL: https://issues.apache.org/jira/browse/FLINK-3779
>             Project: Flink
>          Issue Type: Improvement
>          Components: Distributed Coordination
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
>             Fix For: 1.2.0
>
>
> Flink offers state abstractions for user functions in order to guarantee fault-tolerant
processing of streams. Users can work with both non-partitioned (Checkpointed interface) and
partitioned state (getRuntimeContext().getState(ValueStateDescriptor) and other variants).
> The partitioned state interface provides access to different types of state that are
all scoped to the key of the current input element. This type of state can only be used on
a KeyedStream, which is created via stream.keyBy().
> Currently, all of this state is internal to Flink and used in order to provide processing
guarantees in failure cases (e.g. exactly-once processing).
> The goal of Queryable State is to expose this state outside of Flink by supporting queries
against the partitioned key value state.
> This will help to eliminate the need for distributed operations/transactions with external
systems such as key-value stores which are often the bottleneck in practice. Exposing the
local state to the outside moves a good part of the database work into the stream processor,
allowing both high throughput queries and immediate access to the computed state.
> This is the initial design doc for the feature: https://docs.google.com/document/d/1NkQuhIKYmcprIU5Vjp04db1HgmYSsZtCMxgDi_iTN-g.
Feel free to comment.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message