flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Utkin <cind...@gmail.com>
Subject /newbie/ Share state between streams
Date Fri, 03 Jun 2016 16:34:42 GMT

I am newbie in Flink and have questions about stream states. I can’t find answers in documentation,
but if I just miss one, please link to doc)

1. Is ValueState (and other state classes) are ‘stream' scoped? So that, it is not possible
to share same state with two (or more) different pipelines in a same job:
- sourceA -> keyBy -> mapWithStateXUpdate -> print
- sourceB -> keyBy -> mapUsingStateXValue -> print

Operators ‘mapWithStateXUpdate’ and ‘mapUsingStateXValue’ will use separate copy of
StateX even they use same name for it and same key (ValueStateDescriptor(“StateX”…)).
 Is it right?

2. In Streaming Guide about connect() operation:
Connect allowing for shared state between the two streams.

But how do access state from operators after connect - connect() returns ConnectedStream but
not Keyed, so states defined early is not accessible. Or it means that doing keyBy after connect()
will allow to define new state based on values from both streams? 

3. Is state are ‘operator’ scoped? 

- source -> KeyBy -> mapWithStateX_1 -> keyBy -> mapWithStateX_2

Assume both map try to use state with same name (ValueStateDescriptor(“StateX”…)). But
despite of that, they will have different copies. is it right?


Andrey Utkin <cindrhc@gmail.com <mailto:cindrhc@gmail.com>>

View raw message