flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9269) Concurrency problem in HeapKeyedStateBackend when performing checkpoint async
Date Sat, 28 Apr 2018 03:04:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457332#comment-16457332
] 

ASF GitHub Bot commented on FLINK-9269:
---------------------------------------

GitHub user sihuazhou opened a pull request:

    https://github.com/apache/flink/pull/5934

    [FLINK-9269][state]  fix concurrency problem when performing checkpoint in HeapKeyedStateBackend

    ## What is the purpose of the change
    
    *This PR fixes the concurrency problem in `HeapKeyedStateBackend`, and contains an additional
hotfix for `RocksDBKeyedStateBackend` to duplicate the `KeySerializer` for full checkpoint*
    
    
    ## Brief change log
    
      - *fix concurrency problem when performing checkpoint in `HeapKeyedStateBackend`*
      - *[hotfix] duplicate the key serializer when performing full checkpoint  in `RocksDBKeyedBackend`*
    
    
    ## Verifying this change
    
    This change is a trivial rework / code cleanup without any test coverage.
    ## Does this pull request potentially affect one of the following parts:
    
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (yes)
    
    ## Documentation
    
    no

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sihuazhou/flink FLINK-9269

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5934.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5934
    
----
commit f60d4d1fcd85612c792bec4d0c0d9ba1b142e1e9
Author: sihuazhou <summerleafs@...>
Date:   2018-04-28T02:50:41Z

    fix concurrency problem when performing checkpoint in HeapKeyedStateBackend.

commit c998dbfc78b657bcbd47a836d23b819ce61778b9
Author: sihuazhou <summerleafs@...>
Date:   2018-04-28T02:57:11Z

    [hotfix] duplicate the key serializer when performing full checkpoint in RocksDBKeyedBackend.

----


> Concurrency problem in HeapKeyedStateBackend when performing checkpoint async
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-9269
>                 URL: https://issues.apache.org/jira/browse/FLINK-9269
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.5.0
>            Reporter: Sihua Zhou
>            Assignee: Sihua Zhou
>            Priority: Blocker
>             Fix For: 1.5.0
>
>
> {code:java}
> @Nonnull
> @Override
> protected SnapshotResult<KeyedStateHandle> performOperation() throws Exception
{
>     // do something
>    long[] keyGroupRangeOffsets = new long[keyGroupRange.getNumberOfKeyGroups()];
>    for (int keyGroupPos = 0; keyGroupPos < keyGroupRange.getNumberOfKeyGroups(); ++keyGroupPos)
{
>       int keyGroupId = keyGroupRange.getKeyGroupId(keyGroupPos);
>       keyGroupRangeOffsets[keyGroupPos] = localStream.getPos();
>       outView.writeInt(keyGroupId);
>       for (Map.Entry<String, StateTable<K, ?, ?>> kvState : stateTables.entrySet())
{
>         // do something
>       }
>     }
>     // do something
> }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message