flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StefanRRichter <...@git.apache.org>
Subject [GitHub] flink pull request #5885: [FLINK-8715] Remove usage of StateDescriptor in st...
Date Fri, 27 Apr 2018 14:02:10 GMT
Github user StefanRRichter commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5885#discussion_r184696134
  
    --- Diff: flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBKeyedStateBackend.java
---
    @@ -1116,148 +1115,177 @@ private void restoreKeyGroupsShardWithTemporaryHelperInstance(
     	// ------------------------------------------------------------------------
     
     	/**
    -	 * Creates a column family handle for use with a k/v state. When restoring from a snapshot
    -	 * we don't restore the individual k/v states, just the global RocksDB database and
the
    -	 * list of column families. When a k/v state is first requested we check here whether
we
    -	 * already have a column family for that and return it or create a new one if it doesn't
exist.
    +	 * Registers a k/v state information, which includes its state id, type, RocksDB column
family handle, and serializers.
     	 *
    -	 * <p>This also checks whether the {@link StateDescriptor} for a state matches
the one
    -	 * that we checkpointed, i.e. is already in the map of column families.
    +	 * When restoring from a snapshot, we don’t restore the individual k/v states, just
the global RocksDB database and
    +	 * the list of k/v state information. When a k/v state is first requested we check here
whether we
    +	 * already have a registered entry for that and return it (after some necessary state
compatibility checks)
    +	 * or create a new one if it does not exist.
     	 */
    -	@SuppressWarnings("rawtypes, unchecked")
    -	protected <N, S> ColumnFamilyHandle getColumnFamily(
    -		StateDescriptor<?, S> descriptor, TypeSerializer<N> namespaceSerializer)
throws IOException, StateMigrationException {
    +	private Tuple2<ColumnFamilyHandle, RegisteredKeyedBackendStateMetaInfo<?, ?>>
tryRegisterKvStateInformation(
    --- End diff --
    
    This method is rarely invoked, the tuples are not immutable so giving a defensive copy
can also have its benefits an I feel like this is cleaner than having casts all over the place.
There might also be different ways to solve the general problem, but this feels better to
me than the initial version.


---

Mime
View raw message