flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7822) Add documentation for the new queryable state client.
Date Tue, 07 Nov 2017 11:27:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16241852#comment-16241852
] 

ASF GitHub Bot commented on FLINK-7822:
---------------------------------------

Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4966#discussion_r149337260
  
    --- Diff: docs/dev/stream/state/queryable_state.md ---
    @@ -32,38 +32,67 @@ under the License.
       likely that there will be breaking API changes on the client side in the upcoming Flink
versions.
     </div>
     
    -In a nutshell, this feature allows users to query Flink's managed partitioned state
    -(see [Working with State]({{ site.baseurl }}/dev/stream/state/state.html)) from outside
of
    -Flink. For some scenarios, queryable state thus eliminates the need for distributed
    -operations/transactions with external systems such as key-value stores which are often
the
    -bottleneck in practice.
    +In a nutshell, this feature exposes Flink's managed keyed (partitioned) state
    +(see [Working with State]({{ site.baseurl }}/dev/stream/state/state.html)) to the outside
world and 
    +allows the user to query a job's state from outside Flink. For some scenarios, queryable
state 
    +eliminates the need for distributed operations/transactions with external systems such
as key-value 
    +stores which are often the bottleneck in practice. In addition, this feature may be particularly

    +useful for debugging purposes.
     
     <div class="alert alert-warning">
    -  <strong>Attention:</strong> Queryable state accesses keyed state from a
concurrent thread rather
    -  than synchronizing with the operator and potentially blocking its operation. Since
any state
    -  backend using Java heap space, e.g. MemoryStateBackend or
    -  FsStateBackend, does not work with copies when retrieving values but instead directly
    -  references the stored values, read-modify-write patterns are unsafe and may cause the
    -  queryable state server to fail due to concurrent modifications.
    -  The RocksDBStateBackend is safe from these issues.
    +  <strong>Attention:</strong> When querying a state object, that object is
accessed from a concurrent 
    +  thread without any synchronization or copying. This is a design choice, as any of the
above would lead
    +  to increased job latency, which we wanted to avoid. Since any state backend using Java
heap space, 
    +  <i>e.g.</i> <code>MemoryStateBackend</code> or <code>FsStateBackend</code>,
does not work 
    +  with copies when retrieving values but instead directly references the stored values,
read-modify-write 
    +  patterns are unsafe and may cause the queryable state server to fail due to concurrent
modifications.
    +  The <code>RocksDBStateBackend</code> is safe from these issues.
     </div>
     
    +## Architecture
    +
    +Before showing how to use the Queryable State, it is useful to briefly describe the entities
that compose it.
    +The Queryable State consists of three main entities:
    --- End diff --
    
    nit: "The Queryable State feature"


> Add documentation for the new queryable state client.
> -----------------------------------------------------
>
>                 Key: FLINK-7822
>                 URL: https://issues.apache.org/jira/browse/FLINK-7822
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Documentation, Queryable State
>    Affects Versions: 1.4.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>            Priority: Blocker
>             Fix For: 1.4.0
>
>
> This has to include:
> 1) how to enable queryable state (jars)...
> 2) configuration parameters
> 3) new APIs
> 4) Guarantees/Semantics/Limitations (e.g. memory backend)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message