flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: How does custom object/data structures get mapped into rocksdb underneath?
Date Mon, 09 Mar 2020 13:26:27 GMT
Hi,

when working with state you have to create a state descriptor ([1]). 
This will have a TypeInformation or TypeSerializer that Flink uses to 
(de)serialize your state. This will be used to serialize your state to 
bytes, in the case of RocksDB we serialize both the key (the key of the 
record that is used to key the stream) and the value (your state object, 
MyState in you example) to bytes and store them in the RocksDB key-value 
store.

Best,
Aljoscha

[1] 
https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/state/state.html

On 09.03.20 12:27, kant kodali wrote:
> Hi All,
> 
> I want to do stateful streaming and I was wondering how Custom objects get
> mapped into rocksdb?
> 
> say I have the following class that represents my state
> 
> public class MyState {
>      private HashMap<String, T> map1 ; // T can be any type
>      private HashMap<Integer, S> map2; // S can be any type
> }
> 
> I wonder how these two maps gets mapped into rocksdb? and how does Flink
> know that map1 and map2 together are part of my state but not
> individual ones in isolation?
> 
> Thanks!
> 

Mime
View raw message