flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: streaming restored state after restart
Date Sat, 13 Jun 2020 11:51:49 GMT
Hi Adam,

you could have a local field in your operator which you initialize with
true on creation. Then in the map or process function you could check this
field and if true output all buffered state. At last you only need to set
this field to false so that any subsequent call to the map/process function
won't trigger the emission of the buffered elements in your state.

Cheers,
Till

On Thu, Jun 11, 2020 at 2:58 PM Adam Atrea <atreaadam@gmail.com> wrote:

> Hi,
>
> I'm new to Flink - but after reading the documentation -
>
> What would be the best approach to stream data from a restored state
> following a job restart ?
> Say I have a MapState that gets populated during streaming with various
> computed results within a stateful operator.
>
> Upon job restart or on task failure recovery I am looking to stream the
> data from that restored state downstream so that the operator will replay
> it.
> The data just to be directed to the downstream - at least once - so
> multiple tasks may submit the same data.
>
> Thanks,
>
> Adam
>

Mime
View raw message