flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2664) Remove partitioned states when set to null
Date Mon, 14 Sep 2015 11:27:45 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14743385#comment-14743385
] 

ASF GitHub Bot commented on FLINK-2664:
---------------------------------------

GitHub user gyfora opened a pull request:

    https://github.com/apache/flink/pull/1126

    [FLINK-2664] [streaming] Allow partitioned state removal

    This PR changes the way Partitioned states handle null values.
    
    Before this PR the user was not allowed to put null values in a partitioned state and
an exception was thrown. This did not allow the user to remove the state for the current key
(so the default value will be retreived next time).
    
    Now whenever a user updates a partitioned state with a null value, the system will remove
the state for the respective key from that map containing all the <key,state> pairs
allowing for the garbage collection of unwanted user state.
    
    The StatefulOperatorTest has also been modified to test for the changed behavior.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gyfora/flink 2664

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/1126.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1126
    
----
commit dd40ea508f78c0487e003de2c2cf13cb12681db9
Author: Gyula Fora <gyfora@apache.org>
Date:   2015-09-14T09:13:29Z

    [FLINK-2664] [streaming] Allow partitioned state removal

----


> Remove partitioned states when set to null
> ------------------------------------------
>
>                 Key: FLINK-2664
>                 URL: https://issues.apache.org/jira/browse/FLINK-2664
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming
>            Reporter: Gyula Fora
>            Assignee: Gyula Fora
>
> Currently there is no way to remove a specific key from the partitioned state stored
at streaming operators, which can cause unnecessary state accumulation.
> I suggest to remove the partitioned state for the current input when the state is set
to null, and upon next retrieval simply return the default value. This allows the implementation
of streaming programs that can garbage collect their own unwanted state.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message