flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7683) Add method to iterate over all of the existing keys in a statebackend
Date Tue, 26 Sep 2017 16:00:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180993#comment-16180993
] 

ASF GitHub Bot commented on FLINK-7683:
---------------------------------------

Github user pnowojski commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4722#discussion_r141083304
  
    --- Diff: flink-contrib/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
    @@ -252,6 +257,43 @@ public RocksDBKeyedStateBackend(
     		LOG.debug("Setting initial keyed backend uid for operator {} to {}.", this.operatorIdentifier,
this.backendUID);
     	}
     
    +	@Override
    +	public <N> Stream<K> getKeys(String field, N namespace) {
    +		Tuple2<ColumnFamilyHandle, ?> columnInfo = kvStateInformation.get(field);
    +		if (columnInfo == null) {
    +			return Stream.empty();
    +		}
    +
    +		RocksIterator iterator = db.newIterator(columnInfo.f0);
    +		iterator.seekToFirst();
    +
    +		Iterator<K> sourceIterator = new Iterator<K>() {
    +			@Override
    +			public boolean hasNext() {
    --- End diff --
    
    I've changed this code to iterate over a snapshot of a db, so this should be fine now
for RocksDB.


> Add method to iterate over all of the existing keys in a statebackend
> ---------------------------------------------------------------------
>
>                 Key: FLINK-7683
>                 URL: https://issues.apache.org/jira/browse/FLINK-7683
>             Project: Flink
>          Issue Type: New Feature
>          Components: State Backends, Checkpointing
>            Reporter: Piotr Nowojski
>            Assignee: Piotr Nowojski
>
> This is required to make possible preserving backward compatibility while changing state
definition of a keyed state operator (to do so operator must iterate over all of the existing
keys and rewrites them into a new state variable).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message